Re: [xSocket-develop] [discusstion] applying xSocket to implement an effective and robust file server

[Gregor R. <gre...@go...>]

Hi,

I'm not sure if I understand your code. The DataHander is used to send the
file to the client. On the other side the DataHandler will be used by
handling the data command (which is used to upload data from the client to
the server?)

I added a simple FileServer into the test trunk of xSocket. See
http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleFileServer.javaand
http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleFileServerClient.java

May be it helps you to write your server

Gregor



2008/11/18 beijing welcome <wel...@gm...>

> hi, Gregor,
> many thanks for your suggestion, i tried to do.
>
> the code below were copied from SimpleSmtpClient.java and
> SimpleSmtpServer.java, for some reason, it didn't work.
> i'm in trouble in the filechannel parts, which were comments in *bold*below.
> could you please point me out what's the error?
>
>
> ========== SimplePicClient.java ============
> package picserver;
>
> import java.io.File;
> import java.io.FileInputStream;
> import java.io.IOException;
> import java.io.RandomAccessFile;
> import java.nio.BufferUnderflowException;
> import java.nio.channels.FileChannel;
> import java.util.logging.ConsoleHandler;
> import java.util.logging.Level;
> import java.util.logging.Logger;
>
>
> import org.xsocket.LogFormatter;
> import org.xsocket.MaxReadSizeExceededException;
> import org.xsocket.connection.BlockingConnection;
> import org.xsocket.connection.IBlockingConnection;
>
>
> public final class SimplePicClient {
>
>     private static final Logger LOG =
> Logger.getLogger(SimplePicClient.class.getName());
>
>
>     private String host = null;
>     private int port = -1;
>
>     public static void main(String[] args) throws Exception {
>
>         Logger logger = Logger.getLogger(SimplePicClient.class.getName());
>         logger.setLevel(Level.FINE);
>
>         ConsoleHandler ch = new ConsoleHandler();
>         ch.setLevel(Level.FINE);
>         ch.setFormatter(new LogFormatter());
>         logger.addHandler(ch);
>
>         String host = "127.0.0.1";
>         int port = 9527;
>
>         SimplePicClient smtpClient = new SimplePicClient(host, port);
>         smtpClient.send();
>     }
>
>
>
>     public SimplePicClient(String host, int port) throws IOException {
>         this.host = host;
>         this.port = port;
>     }
>
>
>
>     public void send() throws IOException {
>
>         IBlockingConnection con = new BlockingConnection(host, port);
>
>         // read greeting
>         readResponse(con);
>
>         sendCmd(con, "Helo you");
>         readResponse(con);
>
>         sendCmd(con, "Data");
>         readFileResponse(con);
>         con.write("\r\n.\r\n");
>
>
>         sendCmd(con, "Quit");
>         readResponse(con);
>
>         con.close();
>     }
>
>
>     private void sendCmd(IBlockingConnection con, String cmd) throws
> IOException {
>         LOG.fine("sending " + cmd);
>         con.write(cmd + "\r\n");
>     }
>
>     private String readResponse(IBlockingConnection con) throws IOException
> {
>         String response = con.readStringByDelimiter("\r\n");
>         LOG.fine("receiving " + response);
>
>         return response;
>     }
>
> *    private String readFileResponse(IBlockingConnection con)
>         throws IOException, BufferUnderflowException,
>         MaxReadSizeExceededException {
>
>         int length = con.readInt();
> // the output is:
> // 24:54:06,359 10 FINE [SimplePicClient#readFileResponse] length:
> 1164866661
>         LOG.fine("length: " + length);
>         FileChannel fc = new RandomAccessFile(
>                 "D:\\1\\tmp.txt", "rw").getChannel();
>         fc.transferTo(0, length, con);
>         fc.close();
>         return "Done";
>     }*
> }
>
> ============== SimplePicServer.java =================
> package picserver;
>
> import java.io.File;
> import java.io.IOException;
> import java.io.RandomAccessFile;
> import java.nio.BufferUnderflowException;
> import java.nio.channels.FileChannel;
>
> import org.xsocket.Execution;
> import org.xsocket.MaxReadSizeExceededException;
> import org.xsocket.connection.IConnectHandler;
> import org.xsocket.connection.IDataHandler;
> import org.xsocket.connection.INonBlockingConnection;
> import org.xsocket.connection.IServer;
> import org.xsocket.connection.Server;
> import org.xsocket.connection.ConnectionUtils;
> import org.xsocket.connection.IConnection.FlushMode;
>
>
> public final class SimplePicServer extends Server {
>
>
>     public static void main(String[] args) throws Exception {
>         int port = 9527;
>         String outDir = "c://tmp";
>         if (!new File(outDir).exists()) {
>             System.out.println(outDir + " does not exists. creating
> directory");
>             new File(outDir).mkdirs();
>         }
>
>         System.out.println("writing mails to directory " + outDir);
>
>         IServer server = new SimplePicServer(port, outDir);
>         server.setFlushMode(FlushMode.ASYNC);
> //        server.start();
>         ConnectionUtils.registerMBean(server);
>         server.run();
>     }
>
>
>     public SimplePicServer(int port, String outDir) throws IOException {
>         super(port, new SimplePicHandler(outDir));
>     }
>
>     @Execution(Execution.NONTHREADED)
>     private static final class SimplePicHandler
>         implements IConnectHandler, IDataHandler {
>
>         private String msgFileDir = null;
>         private int countReceivedMessages = 0;
>
>         public SimplePicHandler(String msgFileDir) {
>             this.msgFileDir = msgFileDir;
>         }
>
>
>
>         public boolean onConnect(INonBlockingConnection connection)
>             throws IOException, BufferUnderflowException,
>             MaxReadSizeExceededException {
>             connection.write("220 ready \r\n");
>
>             return true;
>         }
>
>
>         public boolean onData(INonBlockingConnection connection)
>             throws IOException, BufferUnderflowException,
>             MaxReadSizeExceededException {
>
>             String cmd =
> connection.readStringByDelimiter("\r\n").toUpperCase();
>             System.out.println(cmd);
>             if (cmd.startsWith("HELO")) {
>                 connection.write("250 Service\r\n");
>                 System.out.println("when HELO");
>             } else if(cmd.equals("DATA")) {
>                 File msgFile = new File("D:\\1\\mail.txt");
>                 connection.setHandler(
>                         new DataHandler(msgFile, this));
>                 connection.write("Enter message, ending with \".\"\r\n");
>
>                 System.out.println("when DATA");
>
>             } else if (cmd.startsWith("QUIT")) {
>                 connection.write("221 service closing connection\r\n");
>                 connection.close();
>                 System.out.println("when QUIT");
>
>             } else {
>                 connection.write("500 Unrecognized command\r\n");
>                 System.out.println("when else");
>
>             }
>
>             return true;
>         }
>
>
>         String getMessageFileDirectory() {
>             return msgFileDir;
>         }
>
>         void setMessageFileDirectory(String msgFileDir) {
>             this.msgFileDir = msgFileDir;
>         }
>
>         int getCountReceivedMessages() {
>             return countReceivedMessages;
>         }
>
>         void incCountReceiveMessages() {
>             countReceivedMessages++;
>         }
>     }
>
>
>     @Execution(Execution.MULTITHREADED)
>     private static final class DataHandler implements IDataHandler {
>
>         private SimplePicHandler simplePicHandler;
>         private File file;
>         private FileChannel fc;
>
>         public DataHandler(File file, SimplePicHandler simplePicHandler) {
>             this.file = file;
>             this.simplePicHandler = simplePicHandler;
>         }
>
> *        public boolean onData(INonBlockingConnection connection)
>             throws IOException, BufferUnderflowException,
>             MaxReadSizeExceededException {
>
>             if (fc == null) {
>                 fc = new RandomAccessFile(file, "r").getChannel();
>             }
>             int length = (int)fc.size();
>             System.out.println("length: " + length);
>             connection.write(length);
>             connection.transferFrom(fc);
>             fc.close();
>             fc = null;
> // it's not correct, the handler here failed to go back to SimplePicHandler
> // the following command are all dealed by DataHandler, which should be
> dealed by SimplePicHandler.
>             connection.setHandler(simplePicHandler);
>             connection.write("250 OK\r\n");
>             return true;
>         }*
>     }
>
> }
>
>
>
> 2008/11/18 Gregor Roth <gre...@go...>
>
> Hi,
>>
>> in most cases it is better to use the multithreaded mode (multithreaded is
>> the default mode). Running the service in a non-threaded mode is a
>> performance optimization. However, the non-threaded mode will work only
>> work, if the handler doesn't perform I/O operations such as file or network
>> operations.
>>
>> To avoid outstanding threads while receiving file data, you could read the
>> network data and write it to a file in a non-blocking mode. For instance see
>> the Data Handler of the SimpleSmtpServer ->
>> http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleSmtpServer.java
>>
>> To send a file a sync mode has to be used. For instance see the
>> UploadHandler2 of the SimpleFileChannelPerformanceTest >
>> http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleFileChannelPerformanceTest.java
>> With Java 1.7 you will have the option to access the file in an async
>> mode. In this case you can use a non-blocking, async approach similar to the
>> file upload scenario.
>>
>> If the server reaches the limit by serving many clients, a transport level
>> load balancing solution will be used to spread the request over a farm of
>> servers. For a more detailed explanation see link
>> http://www.javaworld.com/javaworld/jw-10-2008/jw-10-load-balancing-1.html
>>
>> Gregor
>>
>>
>>
>>
>>
>> 2008/11/17 beijing welcome <wel...@gm...>
>>
>>>  hi,
>>> a discussion for applying xSocket to implement an effective and robust
>>> file server.
>>>
>>> a TCP server stores lots of small pictures (from 1k to 5k for each size).
>>> a client downloads pictures from server frequently, but seldom do
>>> uploading job.
>>>
>>> as far as i know for xSocket, the server can use
>>> SERVER: transferFrom a FileChannel to BlockingConnection in on
>>> IDataHandler.onData()
>>> CLIENT: can use transferTo a FileChannel from BlockingConnection
>>>
>>> the communication between client and server has two steps:
>>> 1. client sends a query to server, server send back the pictures list
>>> which need to download.
>>> 2. client begins to download the pictures from server.
>>>
>>> if the client need to download more than 10 pictures in a seconds, then
>>> it seems reasonable to use multi thread.
>>>
>>> the question is how to design in server side.
>>> 1) if use multi thread, how to write the IDataHandler.onData()? if it's
>>> BlockingConnection, will it block other threads for this connection? if
>>> NonBlockingConnection, need to write to bytebuffer.duplicate()?
>>>
>>> 2) if many clients connect to the same server (e.g. more than 10k), the
>>> multi thread will rise the connection number? how to minimize the load of
>>> server? if use proxy to do load balance, any other problem?
>>>
>>>
>>>
>>> what's your opinion?
>>>
>>> any comments are appreciated.
>>>
>>> -------------------------------------------------------------------------
>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>>> challenge
>>> Build the coolest Linux based applications with Moblin SDK & win great
>>> prizes
>>> Grand prize is a trip for two to an Open Source event anywhere in the
>>> world
>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>> _______________________________________________
>>> xSocket-develop mailing list
>>> xSo...@li...
>>> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>>>
>>>
>>
>