[Lxr-dev] [ lxr-Bugs-506627 ] Excessive memory usage

[<no...@so...>]

Bugs item #506627, was opened at 2002-01-21 12:48
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=390117&aid=506627&group_id=27350

Category: genxref
>Group: current cvs
Status: Open
Resolution: None
>Priority: 4
Submitted By: Nobody/Anonymous (nobody)
>Assigned to: Malcolm Box (mbox)
Summary: Excessive memory usage

Initial Comment:
LXR uses much more memory than is required. I have a 
large binary with debugging symbols (80+M) in my 
source tree and attempting to index it I get the 
following error.

Out of memory during "large" request for 134221824 
bytes, total sbrk() is 203704320 bytes at 
lib/LXR/Lang.pm line 42.

Note the first value is equal to 128M. A quick look at 
the source code shows the offending line below:

# Try to see if it's a script
$files->getfile($pathname, $release) =~ /^#!\s*(\S+)/s;

And the offending function from Plain.pm:

sub getfile {
        my ($self, $filename, $release) = @_;
        my ($buffer);
        local ($/) = undef;

        open(FILE, $self->toreal($filename, $release)) 
|| return undef;
        $buffer = <FILE>;
        close(FILE);
        return $buffer;
}

Reading the whole file into memory to determine if it 
is a shell script is really excessive. Surely the 
first line should suffice. File::Type may do a better 
job, but it operates only on a filename, and I'm not 
sure about portability.

----------------------------------------------------------------------

>Comment By: Malcolm Box (mbox)
Date: 2002-01-22 23:58

Message:
Logged In: YES 
user_id=215386

I suppose one answer would be "Don't do that then" :-) 
Keeping binaries in a source tree is rarely a good thing.

In terms of fixing this, the easiest way would be to use the
Files::getfilehandle rather than Files::getfile to read the
first line.

----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=390117&aid=506627&group_id=27350