java lzma decode with multithreading(future tasks) - HIGH memory consumption

Help
Tamas
2014-01-10
2014-01-10
  • Tamas
    Tamas
    2014-01-10

    Hello,

    I want to write a small application where I need to decode some lzma files.
    Program is using multithreading so I want decode many files in one moment.

    I am using decoder implementation from last lzma archive from alone.java.

    Everthing works fine for single thread.
    After I want decode files with java future tasks it works well but it has memory leaks.
    I created 24 threads. Each thread has own new Decoder() object.

    After the build application has 2 GB memory consumption. Allocated memory is released only after application is closed.

    How I am using it:

    //multithreading
    int numberOfProcessors = Runtime.getRuntime().availableProcessors();
    ExecutorService executor = Executors.newFixedThreadPool(numberOfProcessors);

    List<Future<Integer>> futureList = new ArrayList<Future<Integer>>();
    List<Decoder> decoderList = new ArrayList<Decoder>();

    //create 24 lzma decoders
    for(int hour = 0; hour<24; hour++) {
    decoderList.add(new Decoder());
    }

    for(int hour = 0; hour<24; hour++) {
    file = new File("file_xy");

    futureList.add(executor.submit(new MyDecoder(file, decoderList.get(hour))));
    }

    for(Future<Integer> futureResult : futureList) {
    futureResult.get(); //wait for result
    }

    executor.shutdown();

    Is it possible use to multithreading decoding?
    Why is it not working?

    I will be glad for any help.

    Thanks,
    Tomas

     
    Last edit: Tamas 2014-01-10
  • Igor Pavlov
    Igor Pavlov
    2014-01-10

    I don't work in Java now. So it's difficult to say for me now.
    Maybe Java's garbage collector is not good?