Few requests through the same NetSocket problem

[Maxim Ryndin]

Hi,

I need to do few request to the same host for different pages. My first aproach was to create one NetSocket variable and use it for this host. But after five or six requests I've got the next excaption:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException
at java.lang.System.arraycopy(Native Method)
at org.silvertunnel_ng.netlib.layer.tor.stream.TCPStreamOutputStream.write(TCPStreamOutputStream.java:170)
at org.silvertunnel_ng.netlib.layer.tor.stream.TCPStreamOutputStream.write(TCPStreamOutputStream.java:166)
at org.silvertunnel_ng.netlib.layer.tor.stream.TCPStreamOutputStream.write(TCPStreamOutputStream.java:181)
at org.silvertunnel_ng.netlib.util.HttpUtil.request(HttpUtil.java:285)
at org.silvertunnel_ng.netlib.util.HttpUtil.get(HttpUtil.java:183)
at tor.TorCheck.download(TorCheck.java:142)
at tor.TorCheck.download(TorCheck.java:158)
at tor.TorCheck.main(TorCheck.java:118)

After this I changed my code to create new NetSocket instance per request to the same host.
It looks like you don't clean buffer and bufferFilled in TCPStreamOutputStream after request processing.

Regards, Max

[Tobias Boese]

Hi Max,

the bufferFilled is set to 0 in the flush method which should be executed in the write method.

Are you running the requests threaded? Maybe this part is not threadsafe, can you show me an example code?

thanks and regards,
Tobi

[Maxim Ryndin]

Hi Tobi,

Here is the code I use:

public static void main(String[] args) throws IOException
    {
        String adress = args.length >= 1 ? args[0] : "";
        int depth = args.length >= 2 ? Integer.parseInt(args[1]) : 2;
        Params params = null;

        // parsing

        TorCheck torcheck = new TorCheck(params, depth);
        if (torcheck.connect()) {
            torcheck.download(1, torcheck.path);
        }
        torcheck.close();
    }
    public boolean connect() {
        try{
            TORCHECK_NETADDRESS = new TcpipNetAddress(serverName, port);
            topSocket = NetFactory.getInstance().getNetLayerById(NetLayerIDs.TOR_OVER_TLS_OVER_TCPIP)
                    .createNetSocket(null, null, TORCHECK_NETADDRESS);
            HttpUtil.getInstance();
            return true;
        }catch(IOException e){
            e.printStackTrace();
        }
        return false;
    }

    public void download(int curDepth, String curPath){
        try {           
            final byte[] httpResponse = HttpUtil.get(socket, TORCHECK_NETADDRESS, curPath, 5000);
            String fileName = dirName + curPath.replaceAll(":|&|[?]", "_") + ".html";
            File file = new File(fileName);
            file.getParentFile().mkdirs();
            FileOutputStream fos = new FileOutputStream(file);
            fos.write(httpResponse);
            fos.flush();
            fos.close();
            processedPathes.add(curPath);
            System.out.println("File '" + fileName + "' downloaded");
            String httpResponseStr = new String(httpResponse, "UTF8");
            if (curDepth < depth){
                Matcher m = p.matcher(httpResponseStr);
                while (m.find()){
                    String newPath = m.group().substring(HREF_LENGTH, m.group().length() - 1);
                    if (!processedPathes.contains(newPath)){
                        download(curDepth + 1, newPath);
                    }
                }
            }
            if (socket != null && socket != topSocket) socket.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

Regards,

Max