Re: [xSocket-develop] [discusstion] applying xSocket to implement an effective and robust file serv
Status: Inactive
Brought to you by:
grro
Menu
▾
▴
Re: [xSocket-develop] [discusstion] applying xSocket to implement an effective and robust file server
|
From: beijing w. <wel...@gm...> - 2008-11-19 15:02:30
|
Gregor, thanks you so much for your great help again!
two minor basic questions about client implements.
1. about BlockingConnectionPool
client {
private final BlockingConnectionPool pool = new BlockingConnectionPool();
public void *read*(filename) {
try {
con = pool.getBlockingConnection(host, port);
// do something...
} catch () {
// the TCP connection is still kept, i mena when this function was
called for many times in for loop,
// it still keep the same connection, not waste resource here. right?
pool.destroy(con);
}
}
void main() {
for (int i = 0; i < size; i++) {
client.*read*(files[i]);
}
}
}
2. if the client is implemented by C++ or other language, any tricky issues
for the connection?
2008/11/19 Gregor Roth <gre...@go...>
> Hi,
>
> I'm not sure if I understand your code. The DataHander is used to send the
> file to the client. On the other side the DataHandler will be used by
> handling the data command (which is used to upload data from the client to
> the server?)
>
> I added a simple FileServer into the test trunk of xSocket. See
> http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleFileServer.javaand
> http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleFileServerClient.java
>
> May be it helps you to write your server
>
> Gregor
>
>
>
> 2008/11/18 beijing welcome <wel...@gm...>
>
> hi, Gregor,
>> many thanks for your suggestion, i tried to do.
>>
>> the code below were copied from SimpleSmtpClient.java and
>> SimpleSmtpServer.java, for some reason, it didn't work.
>> i'm in trouble in the filechannel parts, which were comments in *bold*below.
>> could you please point me out what's the error?
>>
>>
>> ========== SimplePicClient.java ============
>> package picserver;
>>
>> import java.io.File;
>> import java.io.FileInputStream;
>> import java.io.IOException;
>> import java.io.RandomAccessFile;
>> import java.nio.BufferUnderflowException;
>> import java.nio.channels.FileChannel;
>> import java.util.logging.ConsoleHandler;
>> import java.util.logging.Level;
>> import java.util.logging.Logger;
>>
>>
>> import org.xsocket.LogFormatter;
>> import org.xsocket.MaxReadSizeExceededException;
>> import org.xsocket.connection.BlockingConnection;
>> import org.xsocket.connection.IBlockingConnection;
>>
>>
>> public final class SimplePicClient {
>>
>> private static final Logger LOG =
>> Logger.getLogger(SimplePicClient.class.getName());
>>
>>
>> private String host = null;
>> private int port = -1;
>>
>> public static void main(String[] args) throws Exception {
>>
>> Logger logger = Logger.getLogger(SimplePicClient.class.getName());
>> logger.setLevel(Level.FINE);
>>
>> ConsoleHandler ch = new ConsoleHandler();
>> ch.setLevel(Level.FINE);
>> ch.setFormatter(new LogFormatter());
>> logger.addHandler(ch);
>>
>> String host = "127.0.0.1";
>> int port = 9527;
>>
>> SimplePicClient smtpClient = new SimplePicClient(host, port);
>> smtpClient.send();
>> }
>>
>>
>>
>> public SimplePicClient(String host, int port) throws IOException {
>> this.host = host;
>> this.port = port;
>> }
>>
>>
>>
>> public void send() throws IOException {
>>
>> IBlockingConnection con = new BlockingConnection(host, port);
>>
>> // read greeting
>> readResponse(con);
>>
>> sendCmd(con, "Helo you");
>> readResponse(con);
>>
>> sendCmd(con, "Data");
>> readFileResponse(con);
>> con.write("\r\n.\r\n");
>>
>>
>> sendCmd(con, "Quit");
>> readResponse(con);
>>
>> con.close();
>> }
>>
>>
>> private void sendCmd(IBlockingConnection con, String cmd) throws
>> IOException {
>> LOG.fine("sending " + cmd);
>> con.write(cmd + "\r\n");
>> }
>>
>> private String readResponse(IBlockingConnection con) throws
>> IOException {
>> String response = con.readStringByDelimiter("\r\n");
>> LOG.fine("receiving " + response);
>>
>> return response;
>> }
>>
>> * private String readFileResponse(IBlockingConnection con)
>> throws IOException, BufferUnderflowException,
>> MaxReadSizeExceededException {
>>
>> int length = con.readInt();
>> // the output is:
>> // 24:54:06,359 10 FINE [SimplePicClient#readFileResponse] length:
>> 1164866661
>> LOG.fine("length: " + length);
>> FileChannel fc = new RandomAccessFile(
>> "D:\\1\\tmp.txt", "rw").getChannel();
>> fc.transferTo(0, length, con);
>> fc.close();
>> return "Done";
>> }*
>> }
>>
>> ============== SimplePicServer.java =================
>> package picserver;
>>
>> import java.io.File;
>> import java.io.IOException;
>> import java.io.RandomAccessFile;
>> import java.nio.BufferUnderflowException;
>> import java.nio.channels.FileChannel;
>>
>> import org.xsocket.Execution;
>> import org.xsocket.MaxReadSizeExceededException;
>> import org.xsocket.connection.IConnectHandler;
>> import org.xsocket.connection.IDataHandler;
>> import org.xsocket.connection.INonBlockingConnection;
>> import org.xsocket.connection.IServer;
>> import org.xsocket.connection.Server;
>> import org.xsocket.connection.ConnectionUtils;
>> import org.xsocket.connection.IConnection.FlushMode;
>>
>>
>> public final class SimplePicServer extends Server {
>>
>>
>> public static void main(String[] args) throws Exception {
>> int port = 9527;
>> String outDir = "c://tmp";
>> if (!new File(outDir).exists()) {
>> System.out.println(outDir + " does not exists. creating
>> directory");
>> new File(outDir).mkdirs();
>> }
>>
>> System.out.println("writing mails to directory " + outDir);
>>
>> IServer server = new SimplePicServer(port, outDir);
>> server.setFlushMode(FlushMode.ASYNC);
>> // server.start();
>> ConnectionUtils.registerMBean(server);
>> server.run();
>> }
>>
>>
>> public SimplePicServer(int port, String outDir) throws IOException {
>> super(port, new SimplePicHandler(outDir));
>> }
>>
>> @Execution(Execution.NONTHREADED)
>> private static final class SimplePicHandler
>> implements IConnectHandler, IDataHandler {
>>
>> private String msgFileDir = null;
>> private int countReceivedMessages = 0;
>>
>> public SimplePicHandler(String msgFileDir) {
>> this.msgFileDir = msgFileDir;
>> }
>>
>>
>>
>> public boolean onConnect(INonBlockingConnection connection)
>> throws IOException, BufferUnderflowException,
>> MaxReadSizeExceededException {
>> connection.write("220 ready \r\n");
>>
>> return true;
>> }
>>
>>
>> public boolean onData(INonBlockingConnection connection)
>> throws IOException, BufferUnderflowException,
>> MaxReadSizeExceededException {
>>
>> String cmd =
>> connection.readStringByDelimiter("\r\n").toUpperCase();
>> System.out.println(cmd);
>> if (cmd.startsWith("HELO")) {
>> connection.write("250 Service\r\n");
>> System.out.println("when HELO");
>> } else if(cmd.equals("DATA")) {
>> File msgFile = new File("D:\\1\\mail.txt");
>> connection.setHandler(
>> new DataHandler(msgFile, this));
>> connection.write("Enter message, ending with \".\"\r\n");
>>
>> System.out.println("when DATA");
>>
>> } else if (cmd.startsWith("QUIT")) {
>> connection.write("221 service closing connection\r\n");
>> connection.close();
>> System.out.println("when QUIT");
>>
>> } else {
>> connection.write("500 Unrecognized command\r\n");
>> System.out.println("when else");
>>
>> }
>>
>> return true;
>> }
>>
>>
>> String getMessageFileDirectory() {
>> return msgFileDir;
>> }
>>
>> void setMessageFileDirectory(String msgFileDir) {
>> this.msgFileDir = msgFileDir;
>> }
>>
>> int getCountReceivedMessages() {
>> return countReceivedMessages;
>> }
>>
>> void incCountReceiveMessages() {
>> countReceivedMessages++;
>> }
>> }
>>
>>
>> @Execution(Execution.MULTITHREADED)
>> private static final class DataHandler implements IDataHandler {
>>
>> private SimplePicHandler simplePicHandler;
>> private File file;
>> private FileChannel fc;
>>
>> public DataHandler(File file, SimplePicHandler simplePicHandler) {
>> this.file = file;
>> this.simplePicHandler = simplePicHandler;
>> }
>>
>> * public boolean onData(INonBlockingConnection connection)
>> throws IOException, BufferUnderflowException,
>> MaxReadSizeExceededException {
>>
>> if (fc == null) {
>> fc = new RandomAccessFile(file, "r").getChannel();
>> }
>> int length = (int)fc.size();
>> System.out.println("length: " + length);
>> connection.write(length);
>> connection.transferFrom(fc);
>> fc.close();
>> fc = null;
>> // it's not correct, the handler here failed to go back to
>> SimplePicHandler
>> // the following command are all dealed by DataHandler, which should be
>> dealed by SimplePicHandler.
>> connection.setHandler(simplePicHandler);
>> connection.write("250 OK\r\n");
>> return true;
>> }*
>> }
>>
>> }
>>
>>
>>
>> 2008/11/18 Gregor Roth <gre...@go...>
>>
>> Hi,
>>>
>>> in most cases it is better to use the multithreaded mode (multithreaded
>>> is the default mode). Running the service in a non-threaded mode is a
>>> performance optimization. However, the non-threaded mode will work only
>>> work, if the handler doesn't perform I/O operations such as file or network
>>> operations.
>>>
>>> To avoid outstanding threads while receiving file data, you could read
>>> the network data and write it to a file in a non-blocking mode. For instance
>>> see the Data Handler of the SimpleSmtpServer ->
>>> http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleSmtpServer.java
>>>
>>> To send a file a sync mode has to be used. For instance see the
>>> UploadHandler2 of the SimpleFileChannelPerformanceTest >
>>> http://xsocket.svn.sourceforge.net/viewvc/xsocket/xsocket/core/trunk/src/test/java/org/xsocket/connection/SimpleFileChannelPerformanceTest.java
>>> With Java 1.7 you will have the option to access the file in an async
>>> mode. In this case you can use a non-blocking, async approach similar to the
>>> file upload scenario.
>>>
>>> If the server reaches the limit by serving many clients, a transport
>>> level load balancing solution will be used to spread the request over a farm
>>> of servers. For a more detailed explanation see link
>>> http://www.javaworld.com/javaworld/jw-10-2008/jw-10-load-balancing-1.html
>>>
>>> Gregor
>>>
>>>
>>>
>>>
>>>
>>> 2008/11/17 beijing welcome <wel...@gm...>
>>>
>>>> hi,
>>>> a discussion for applying xSocket to implement an effective and robust
>>>> file server.
>>>>
>>>> a TCP server stores lots of small pictures (from 1k to 5k for each
>>>> size).
>>>> a client downloads pictures from server frequently, but seldom do
>>>> uploading job.
>>>>
>>>> as far as i know for xSocket, the server can use
>>>> SERVER: transferFrom a FileChannel to BlockingConnection in on
>>>> IDataHandler.onData()
>>>> CLIENT: can use transferTo a FileChannel from BlockingConnection
>>>>
>>>> the communication between client and server has two steps:
>>>> 1. client sends a query to server, server send back the pictures list
>>>> which need to download.
>>>> 2. client begins to download the pictures from server.
>>>>
>>>> if the client need to download more than 10 pictures in a seconds, then
>>>> it seems reasonable to use multi thread.
>>>>
>>>> the question is how to design in server side.
>>>> 1) if use multi thread, how to write the IDataHandler.onData()? if it's
>>>> BlockingConnection, will it block other threads for this connection? if
>>>> NonBlockingConnection, need to write to bytebuffer.duplicate()?
>>>>
>>>> 2) if many clients connect to the same server (e.g. more than 10k), the
>>>> multi thread will rise the connection number? how to minimize the load of
>>>> server? if use proxy to do load balance, any other problem?
>>>>
>>>>
>>>>
>>>> what's your opinion?
>>>>
>>>> any comments are appreciated.
>>>>
>>>>
>>>> -------------------------------------------------------------------------
>>>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>>>> challenge
>>>> Build the coolest Linux based applications with Moblin SDK & win great
>>>> prizes
>>>> Grand prize is a trip for two to an Open Source event anywhere in the
>>>> world
>>>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>>>> _______________________________________________
>>>> xSocket-develop mailing list
>>>> xSo...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/xsocket-develop
>>>>
>>>>
>>>
>>
> |
