Saros - Distributed Party Programming / Bugs / #785 Out of Memory when adding a large amount of files

#785 Out of Memory when adding a large amount of files
Milestone: None
Status: closed-migrated
Owner: nobody
Labels: Network Layer (9)
Priority: 7
Updated: 2018-03-27
Created: 2013-02-03
Creator: Stefan Rossbach
Private: No
Adding a large amount of files during a session can result in OutOfMemory Exceptions in multiple places.
1. in the Sequencer which buffers activities
2. in the binary channel which has now flow control and just pull up every received packet to the dispatcher
ERROR 16:14:51,266 [Bytestream-Extension-receiver-15] (Utils.java:328) Internal Fatal Error:
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.io.ByteArrayOutputStream.toByteArray(Unknown Source)
    at de.fu_berlin.inf.dpp.util.Utils.inflate(Utils.java:546)
    at de.fu_berlin.inf.dpp.net.internal.BinaryChannelTransferObject.accept(BinaryChannelTransferObject.java:100)
    at de.fu_berlin.inf.dpp.net.internal.DataTransferManager$LoggingTransferObject.accept(DataTransferManager.java:184)
    at de.fu_berlin.inf.dpp.net.internal.XMPPReceiver.processTransferObjectToPacket(XMPPReceiver.java:247)
    at de.fu_berlin.inf.dpp.net.internal.XMPPReceiver.access$4(XMPPReceiver.java:231)
    at de.fu_berlin.inf.dpp.net.internal.XMPPReceiver$3$1.run(XMPPReceiver.java:145)
    at de.fu_berlin.inf.dpp.util.Utils$6.run(Utils.java:323)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
ERROR 16:14:51,272 [Bytestream-Extension-receiver-15] (Utils.java:329) Original caller:
StackTrace:
    at de.fu_berlin.inf.dpp.util.Utils.wrapSafe(Utils.java:317)
    at de.fu_berlin.inf.dpp.net.internal.XMPPReceiver$3.run(XMPPReceiver.java:141)
    at de.fu_berlin.inf.dpp.util.Utils$6.run(Utils.java:323)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
ERROR 16:14:51,273 [Bytestream-Extension-receiver-15] (STFController.java:103) uncaught exception in thread: Thread[Bytestream-Extension-receiver-15,5,main]
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.io.ByteArrayOutputStream.toByteArray(Unknown Source)
    at de.fu_berlin.inf.dpp.util.Utils.inflate(Utils.java:546)
    at de.fu_berlin.inf.dpp.net.internal.BinaryChannelTransferObject.accept(BinaryChannelTransferObject.java:100)
    at de.fu_berlin.inf.dpp.net.internal.DataTransferManager$LoggingTransferObject.accept(DataTransferManager.java:184)
    at de.fu_berlin.inf.dpp.net.internal.XMPPReceiver.processTransferObjectToPacket(XMPPReceiver.java:247)
    at de.fu_berlin.inf.dpp.net.internal.XMPPReceiver.access$4(XMPPReceiver.java:231)
    at de.fu_berlin.inf.dpp.net.internal.XMPPReceiver$3$1.run(XMPPReceiver.java:145)
    at de.fu_berlin.inf.dpp.util.Utils$6.run(Utils.java:323)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
ERROR 16:15:03,206 [Timer-10] (Utils.java:328) Internal Fatal Error:
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuffer.append(Unknown Source)
    at com.thoughtworks.xstream.core.util.Base64Encoder.encode(Base64Encoder.java:64)
    at com.thoughtworks.xstream.converters.extended.EncodedByteArrayConverter.toString(EncodedByteArrayConverter.java:77)
    at com.thoughtworks.xstream.converters.extended.EncodedByteArrayConverter.marshal(EncodedByteArrayConverter.java:43)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:67)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:82)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:189)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:180)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:143)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:111)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:67)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:67)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:82)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:189)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:180)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:143)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:111)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:67)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:67)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:82)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:189)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:180)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:139)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:111)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:67)
ERROR 16:15:03,216 [Timer-10] (Utils.java:329) Original caller:
StackTrace:
    at de.fu_berlin.inf.dpp.util.Utils.wrapSafe(Utils.java:317)
    at de.fu_berlin.inf.dpp.util.Utils.runSafeSync(Utils.java:388)
    at de.fu_berlin.inf.dpp.net.internal.ActivitySequencer$2.run(ActivitySequencer.java:534)
    at java.util.TimerThread.mainLoop(Unknown Source)
    at java.util.TimerThread.run(Unknown Source)
ERROR 16:15:03,216 [Timer-10] (STFController.java:103) uncaught exception in thread: Thread[Timer-10,6,main]
java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.ensureCapacityInternal(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuffer.append(Unknown Source)
    at com.thoughtworks.xstream.core.util.Base64Encoder.encode(Base64Encoder.java:64)
    at com.thoughtworks.xstream.converters.extended.EncodedByteArrayConverter.toString(EncodedByteArrayConverter.java:77)
    at com.thoughtworks.xstream.converters.extended.EncodedByteArrayConverter.marshal(EncodedByteArrayConverter.java:43)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:67)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:82)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:189)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:180)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:143)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:111)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:67)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:67)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:82)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:189)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:180)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:143)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:111)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:67)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:67)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:82)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:189)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:180)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:139)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:111)
    at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:67)
Discussion

tobous - 2018-03-22

Might be connected to #380.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
tobous - 2018-03-27

Status: open --> closed-migrated
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Out of Memory when adding a large amount of files

Group

Searches

Help

#785 Out of Memory when adding a large amount of files

Discussion
