ncea: ERROR nco_malloc() unable to allocate

Developers
IITM
2011-11-18
2013-10-17
  • IITM

    IITM - 2011-11-18

    Can we have a solution to the below mentioned problem related to NCO malloc failure.

    ncea: ERROR nco_malloc() unable to allocate 970467840 bytes
    ncea: INFO NCO has reported a malloc() failure. malloc() failures usually indicate that your machine does not have enough free memory (RAM+swap) to perform the requested operation. As such, malloc() failures result from the physical limitations imposed by your hardware. Read http://nco.sf.net/nco.html#mmr for a description of NCO memory usage. There are two workarounds in this scenario. One is to process your data in smaller chunks. The other is to use a machine with more free memory.

    Large tasks may uncover memory leaks in NCO. This is likeliest to occur with ncap. ncap scripts are completely dynamic and may be of arbitrary length and complexity. A script that contains many thousands of operations may uncover a slow memory leak even though each single operation consumes little additional memory. Memory leaks are usually identifiable by their memory usage signature. Leaks cause peak memory usage to increase monotonically with time regardless of script complexity. Slow leaks are very difficult to find. Sometimes a malloc() failure is the only noticeable clue to their existance. If you have good reasons to believe that your malloc() failure is ultimately due to an NCO memory leak (rather than inadequate RAM on your system), then we would be very interested in receiving a detailed bug report.

     
  • Charlie Zender

    Charlie Zender - 2011-11-19

    As the message says "There are two workarounds in this scenario. One is to process your data in smaller chunks. The other is to use a machine with more free memory."  Of course a longer term solution would be to solve this in software, but that would entail a significant code re-write, i.e., require dedicated time/funding.
    cz

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks