|
From: <Dan...@ba...> - 2008-11-04 16:38:48
|
Hi, The program I am debugging does a large mmap() Some small mmap() calls seem to work, but the large mmap(), for an 80GB file, results in EINVAL I've verified that the offset is 0 and the mapping size is a multiple of getpagesize(). The mmap() does work when not using valgrind. Is there some kind of mmap() restriction or size limit with valgrind? I searched for mmap in the documentation and didn't find anything. Is there anything I can do to get valgrind to work with this program, or an alternative tool I can use? The reason for using valgrind is to locate a memory leak. Regards, Daniel _______________________________________________ This e-mail may contain information that is confidential, privileged or otherwise protected from disclosure. If you are not an intended recipient of this e-mail, do not duplicate or redistribute it by any means. Please delete it and any attachments and notify the sender that you have received it in error. Unless specifically indicated, this e-mail is not an offer to buy or sell or a solicitation to buy or sell any securities, investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Barclays. Any views or opinions presented are solely those of the author and do not necessarily represent those of Barclays. This e-mail is subject to terms available at the following link: www.barcap.com/emaildisclaimer. By messaging with Barclays you consent to the foregoing. Barclays Capital is the investment banking division of Barclays Bank PLC, a company registered in England (number 1026167) with its registered office at 1 Churchill Place, London, E14 5HP. This email may relate to or be sent from other members of the Barclays Group. _______________________________________________ |
|
From: John R.
|
Dan...@ba... wrote:
> The program I am debugging does a large mmap()
>
> Some small mmap() calls seem to work, but the large mmap(), for an 80GB
> file, results in EINVAL
>
> I've verified that the offset is 0 and the mapping size is a multiple of
> getpagesize(). The mmap() does work when not using valgrind.
>
> Is there some kind of mmap() restriction or size limit with valgrind? I
> searched for mmap in the documentation and didn't find anything.
The restriction is that there is only one address space that must hold
both the application data and the valgrind(memcheck) data.
When the mmap() fails with EINVAL, then look at the address space using
cat /proc/<pid>/maps
and see if there is 80MB contiguous space available. What other things
are notable about the memory usage as shown by /proc/<pid>/maps ?
(Many threads, many "holes", many large blocks, many shared libs, ...?)
If you are running on i686 with the common allocation of 3GB of address
space for user mode and 1GB for the Linux kernel, then try running the
same program (same binary object file) on x86_64 under a 64-bit kernel
that supports the execution of 32-bit user-mode programs. Most common
Linux distributions provide such support. In this mode you often get
almost 4GB of address space that is accessible by a 32-bit user program,
which may be enough more for your purpose.
If the application is portable to a native 64-bit environment, then
compile and run the app in that environment. There is more address
space, and the application's memory leak is likely to persist.
--
|
|
From: Nicholas N. <nj...@cs...> - 2008-11-04 23:12:16
|
On Tue, 4 Nov 2008, John Reiser wrote: >> The program I am debugging does a large mmap() >> >> Some small mmap() calls seem to work, but the large mmap(), for an 80GB >> file, results in EINVAL >> >> I've verified that the offset is 0 and the mapping size is a multiple of >> getpagesize(). The mmap() does work when not using valgrind. >> >> Is there some kind of mmap() restriction or size limit with valgrind? I >> searched for mmap in the documentation and didn't find anything. > > The restriction is that there is only one address space that must hold > both the application data and the valgrind(memcheck) data. > > When the mmap() fails with EINVAL, then look at the address space using > cat /proc/<pid>/maps > and see if there is 80MB contiguous space available. What other things > are notable about the memory usage as shown by /proc/<pid>/maps ? > (Many threads, many "holes", many large blocks, many shared libs, ...?) > > If you are running on i686 with the common allocation of 3GB of address > space for user mode and 1GB for the Linux kernel, then try running the > same program (same binary object file) on x86_64 under a 64-bit kernel > that supports the execution of 32-bit user-mode programs. Most common > Linux distributions provide such support. In this mode you often get > almost 4GB of address space that is accessible by a 32-bit user program, > which may be enough more for your purpose. > > If the application is portable to a native 64-bit environment, then > compile and run the app in that environment. There is more address > space, and the application's memory leak is likely to persist. He said 80GB, not 80MB, so I assume it's already on x86-64. That's a pretty big file -- not much in a 64-bit address space, admittedly -- but it may be big enough to cause Valgrind problems, not sure. Nick |
|
From: John R.
|
>>> Some small mmap() calls seem to work, but the large mmap(), for an 80GB >>> file, results in EINVAL > He said 80GB, not 80MB, so I assume it's already on x86-64. That's a pretty > big file -- not much in a 64-bit address space, admittedly -- but it may be > big enough to cause Valgrind problems, not sure. Oops, you're right. I mis-read the size as 80 megabytes instead of 80 gigabytes. It still might help to inspect /proc/<pid>/maps. Also /proc/meminfo (to see the general state of system memory), and an actual strace of the app running under memcheck leading up to the failure (to see what valgrind's address space manager actually did.) -- |
|
From: Ashley P. <api...@co...> - 2008-11-05 13:06:25
|
On Tue, 2008-11-04 at 14:58 -0800, John Reiser wrote: > When the mmap() fails with EINVAL, then look at the address space using > cat /proc/<pid>/maps > and see if there is 80MB contiguous space available. Doesn't mmap() return ENOSPC in this case rather than EINVAL? Ashley, |
|
From: John R.
|
>> When the mmap() fails with EINVAL, then look at the address space using >> cat /proc/<pid>/maps >> and see if there is 80MB contiguous space available. > Doesn't mmap() return ENOSPC in this case rather than EINVAL? The manual page says ENOMEM; the code in linux/mm/mmap.c has no "ENOSPC". But things have already "gone wrong" when an 80GB request is not honored on x86_64, so it may be worthwhile to consider other "impossible" conditions. -- |