Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#906 Memory Leak using RxSock

v4.0.1
closed
Mark Miesfeld
APIs (66)
5
2012-08-14
2010-04-24
RodD
No

I am running a Rexx Tcp/Ip program that appears to have a memory leak. I encountered this problem first on 3.2 on 32-bit Windows XP, then installed 4.01 and got the same problem.

While executing a SockRecv loop to receive about 232 megs of data accumulated in a single string, on a Windows XP 32-bit system it eventually runs out of memory. I then ran the program on a 64-bit Windows 7 system (using 64-bit OORexx) and monitored it using Task Manager. I discovered that it used approximately 10.5 Gigs of memory at its peak before finishing with a normal result.

The code is a simple loop that does a SockRecv and then concatenates the data as shown below:

outdata = ''
do while remaining > 0
NumBytes = SockRecv(SockNum, 'InData', remaining)
if NumBytes <= 0 then leave
outdata = outdata || InData
remaining = remaining - NumBytes
end

Eventually it crashes on a 32-bit system at: outdata = outdata || InData

Is there any way to figure out what is causing, and ultimately fixing, the problem? I cannot routinely use a 64-bit system since I had to borrow one for my 64-bit test.

Discussion

  • I wonder if that's just normal for rexx variable processing ie nothing to do with RxSock?

    do n
    blah = blah || somemore
    end

    means storage for blah continually has to grow; maybe oorexx allocates more store without freeing the old stuff often enough?

    How about

    blah = copies(" ", 25010241024)
    blah = ""

    in front of the loop?

     
  • Rick McGuire
    Rick McGuire
    2010-04-24

    I'm quite certain it is. Continually concatenating a string value like that forces the runtime to constantly add additional storage segments to the memory heap to be able to allocate a continguous area large enough for the new result. A better memory approach would be to use a mutablebuffer as the accumulator, particularly if you can determine the size (or an approximate size) ahead of time. This eliminates the need to constantly allocate larger and larger pieces of memory.

     
  • RodD
    RodD
    2010-04-26

    Switching to using a MutableBuffer instead of a string solved the problem. My peak memory usage was just under 900 megs, so it was able to finish on 32-bit Windows.

    But I still wonder if string support should be changed to make better use of memory pools to avoid severe memory fragmentation, which is probably what happened with my original code.

     
  • Mark Miesfeld
    Mark Miesfeld
    2010-08-10

    Since this was really not a rxsock bug, and not really a bug to begin with, I'm closing it as invalid.

    Continually concatenating a string value is known to require more and more memory. If the required memory goes beyond what is available in the system, then ooRexx runs out of memory. Running the program on a 64-bit system showed that the program did actually work.

     


Anonymous


Cancel   Add attachments