|
From: J W <jwd...@go...> - 2011-06-26 20:30:39
|
Hi,
I have a small program which mmaps /dev/fbdev. It runs fine without VG
but when started within, the mmap call failed.
I checked both runs using strace and saw that when run within VG the
parameters passed down to mmap2 are different, and even worse cause
the program to fail ;)
w/o VG:
open("/dev/fb0", O_RDWR) = 3
ioctl(3, FBIOGET_VSCREENINFO, 0xbec2e650) = 0
mmap2(NULL, 1228800, PROT_WRITE, MAP_SHARED, 3, 0) = 0x4016c000
vs (omitting lot of cacheflush calls):
open("/dev/fb0", O_RDWR) = 3
ioctl(3, FBIOGET_VSCREENINFO, 0xbddc05a0) = 0
mmap2(0x497b000, 1228800, PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1
EINVAL (Invalid argument)
If someone has some quick hints how to debug this down, I would be
glad to hear about.
(I am using valgrind svn trunk, running on ARMv6 though, glibc 2.7).
Thanks!
Joerg
|
|
From: Tom H. <to...@co...> - 2011-06-26 23:07:16
|
On 26/06/11 21:30, J W wrote:
> w/o VG:
> open("/dev/fb0", O_RDWR) = 3
> ioctl(3, FBIOGET_VSCREENINFO, 0xbec2e650) = 0
> mmap2(NULL, 1228800, PROT_WRITE, MAP_SHARED, 3, 0) = 0x4016c000
>
> vs (omitting lot of cacheflush calls):
> open("/dev/fb0", O_RDWR) = 3
> ioctl(3, FBIOGET_VSCREENINFO, 0xbddc05a0) = 0
> mmap2(0x497b000, 1228800, PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1
> EINVAL (Invalid argument)
>
> If someone has some quick hints how to debug this down, I would be
> glad to hear about.
Using MAP_FIXED is almost always a really, really bad idea because it
will (normally) forcibly unmap anything which happens to be mapped at
the specified address with no warning.
I believe what is likely to have happened here is that you have tried to
use an address that valgrind is already using and it has protected you
by refusing the mmap rather than allowing you to destroy part of it's
internals by mapping over the top of them.
Tom
--
Tom Hughes (to...@co...)
http://compton.nu/
|
|
From: John R. <jr...@bi...> - 2011-06-27 00:01:41
|
On 06/26/2011 04:07 PM, Tom Hughes wrote:
> On 26/06/11 21:30, J W wrote:
>
>> w/o VG:
>> open("/dev/fb0", O_RDWR) = 3
>> ioctl(3, FBIOGET_VSCREENINFO, 0xbec2e650) = 0
>> mmap2(NULL, 1228800, PROT_WRITE, MAP_SHARED, 3, 0) = 0x4016c000
>>
>> vs (omitting lot of cacheflush calls):
>> open("/dev/fb0", O_RDWR) = 3
>> ioctl(3, FBIOGET_VSCREENINFO, 0xbddc05a0) = 0
>> mmap2(0x497b000, 1228800, PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1
>> EINVAL (Invalid argument)
>>
>> If someone has some quick hints how to debug this down, I would be
>> glad to hear about.
>
> Using MAP_FIXED is almost always a really, really bad idea because it
> will (normally) forcibly unmap anything which happens to be mapped at
> the specified address with no warning.
It seems to me that the differing parameters are documented in
coregrind/m_aspacemgr/aspacemgr-linux.c:
The fundamental mode of operation of aspacem is to support client
mmaps. Here's what happens (in ML_(generic_PRE_sys_mmap)):
[snip]
* Control returns to the wrapper. If VG_(am_get_advisory) has
declared that the map should fail, then it must be made to do so.
Usually, though, the request is considered acceptable, in which
case an "advised" address is supplied. The advised address
replaces the original address supplied by the client, and
MAP_FIXED is set.
So turn on debug logging in aspacem. Also, determine _why_ the mmap2()
got EINVAL: what specific thing is the kernel complaining about?
[What was /proc/self/maps at the time?]
--
|
|
From: John R. <jr...@bi...> - 2011-06-27 01:12:50
|
> w/o VG:
> open("/dev/fb0", O_RDWR) = 3
> ioctl(3, FBIOGET_VSCREENINFO, 0xbec2e650) = 0
> mmap2(NULL, 1228800, PROT_WRITE, MAP_SHARED, 3, 0) = 0x4016c000
>
> vs (omitting lot of cacheflush calls):
> open("/dev/fb0", O_RDWR) = 3
> ioctl(3, FBIOGET_VSCREENINFO, 0xbddc05a0) = 0
> mmap2(0x497b000, 1228800, PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1
> EINVAL (Invalid argument)
Looking at the file name, "/dev/fb0" probably is a hardware device.
A device can have quirks that plain RAM does not have. Get the
kernel source code for the device driver, and look at its mmap routine.
Learn what all the different 'return' statements mean, and how they
influence the result returned by mmap2().
One of the quirks might be address placement or alignment. Asking for
NULL lets the driver choose; you got 0x4016c000. Asking for 0x497b000
is a different alignment modulo 64K; this might matter. If it does,
then you must convince valgrind to ask for the proper alignment.
valgrind does not want to care, so you must enhance your code.
Ask mmap for plain-old RAM that is 60KB larger than what you want,
and with MAP_ANON|MAP_PRIVATE and -1==fd. Take the result, do the alignment
yourself (ignoring ["discarding"] up to 60KB on the low end), then ask mmap
for that address with MAP_FIXED and fd of /dev/fb0.
One of the quirks might be flags. MAP_SHARED might not be allowed,
so try MAP_PRIVATE. If the kernel really does map the device into
your memory space, and if the device is local to the machine you are
on [mmap never was intended to work over a network such as NFS],
then MAP_PRIVATE won't matter: if a write into the mapped address
space succeeds, then the frame buffer will display the result,
regardless of whether the kernel has the region marked as MAP_SHARED
or MAP_PRIVATE. In some ways MAP_SHARED might matter only for
a file system; you are manipulating hardware.
These are two possibilities. Study the device driver mmap routine
to discover others.
--
|
|
From: J W <jwd...@go...> - 2011-06-27 07:03:16
|
Hi,
thanks for plenty of comments.
The parameters I pass down to mmap are the ones given by the first
(plain) run, as shown by strace. They are hard-coded.
The ones VG creates out of them are off, so I was wondering why a VG
run turns a NULL argument into some specific location and changes the
parameters of the syscall.
If this behaviour of VG is by design, and if the address generated by
VG is sane, then I can check, why the driver does not like to operate
on a fixed mapping. But before I am going to do this I just wanted to
make sure that this is not an issue of VG itself.
I will give the logging in aspacem a try, thanks!
Joerg
2011/6/27 John Reiser <jr...@bi...>:
>> w/o VG:
>> open("/dev/fb0", O_RDWR) = 3
>> ioctl(3, FBIOGET_VSCREENINFO, 0xbec2e650) = 0
>> mmap2(NULL, 1228800, PROT_WRITE, MAP_SHARED, 3, 0) = 0x4016c000
>>
>> vs (omitting lot of cacheflush calls):
>> open("/dev/fb0", O_RDWR) = 3
>> ioctl(3, FBIOGET_VSCREENINFO, 0xbddc05a0) = 0
>> mmap2(0x497b000, 1228800, PROT_WRITE, MAP_SHARED|MAP_FIXED, 3, 0) = -1
>> EINVAL (Invalid argument)
>
> Looking at the file name, "/dev/fb0" probably is a hardware device.
> A device can have quirks that plain RAM does not have. Get the
> kernel source code for the device driver, and look at its mmap routine.
> Learn what all the different 'return' statements mean, and how they
> influence the result returned by mmap2().
>
> One of the quirks might be address placement or alignment. Asking for
> NULL lets the driver choose; you got 0x4016c000. Asking for 0x497b000
> is a different alignment modulo 64K; this might matter. If it does,
> then you must convince valgrind to ask for the proper alignment.
> valgrind does not want to care, so you must enhance your code.
> Ask mmap for plain-old RAM that is 60KB larger than what you want,
> and with MAP_ANON|MAP_PRIVATE and -1==fd. Take the result, do the alignment
> yourself (ignoring ["discarding"] up to 60KB on the low end), then ask mmap
> for that address with MAP_FIXED and fd of /dev/fb0.
>
> One of the quirks might be flags. MAP_SHARED might not be allowed,
> so try MAP_PRIVATE. If the kernel really does map the device into
> your memory space, and if the device is local to the machine you are
> on [mmap never was intended to work over a network such as NFS],
> then MAP_PRIVATE won't matter: if a write into the mapped address
> space succeeds, then the frame buffer will display the result,
> regardless of whether the kernel has the region marked as MAP_SHARED
> or MAP_PRIVATE. In some ways MAP_SHARED might matter only for
> a file system; you are manipulating hardware.
>
> These are two possibilities. Study the device driver mmap routine
> to discover others.
>
> --
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Valgrind-developers mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-developers
>
|
|
From: Tom H. <to...@co...> - 2011-06-27 07:47:13
|
On 27/06/11 08:03, J W wrote: > The parameters I pass down to mmap are the ones given by the first > (plain) run, as shown by strace. They are hard-coded. > The ones VG creates out of them are off, so I was wondering why a VG > run turns a NULL argument into some specific location and changes the > parameters of the syscall. Because valgrind needs to control and track all memory allocations so that it can maintain a parallel memory area that records what is and isn't defined so it intercepts all mmap calls and makes it's own decision as to where it wants to place the memory and then makes a MAP_FIXED call to the kernel. > If this behaviour of VG is by design, and if the address generated by > VG is sane, then I can check, why the driver does not like to operate > on a fixed mapping. But before I am going to do this I just wanted to > make sure that this is not an issue of VG itself. As John said it sounds like the problem that valgrind is choosing is not acceptable to the driver for some reason. Most likely it wants some specific alignment that valgrind isn't meeting. Tom -- Tom Hughes (to...@co...) http://compton.nu/ |
|
From: Julian S. <js...@ac...> - 2011-06-27 08:28:42
|
> As John said it sounds like the problem that valgrind is choosing is not > acceptable to the driver for some reason. Most likely it wants some > specific alignment that valgrind isn't meeting. It might be an alignment problem, true. But if it is alignment related, then the alignment must be imposed by the kernel, since the without-V mmap call doesn't specify an address nor MAP_FIXED: mmap2(NULL, 1228800, PROT_WRITE, MAP_SHARED, 3, 0) = 0x4016c000 so if special alignment needs to be provided, this must happen via kernel logic which identifies the fd as referring to /dev/db0. Anyway, the returned value 0x4016c000 is 4-page aligned. Whereas with V we are requesting 0x497b000, which is only 1-page aligned. J W, one kludge you might like to try is to put your program's mmap call inside a loop, which retries up to 16 times. After each failure, do an anonymous mmap of 1 page before retrying. This might just give the effect of making Valgrind eventually try out an address with the alignment you really need. If that makes any sense. J |
|
From: Tom H. <to...@co...> - 2011-06-27 08:32:27
|
On 27/06/11 09:27, Julian Seward wrote: > >> As John said it sounds like the problem that valgrind is choosing is not >> acceptable to the driver for some reason. Most likely it wants some >> specific alignment that valgrind isn't meeting. > > It might be an alignment problem, true. But if it is alignment related, > then the alignment must be imposed by the kernel, since the without-V > mmap call doesn't specify an address nor MAP_FIXED: > > mmap2(NULL, 1228800, PROT_WRITE, MAP_SHARED, 3, 0) = 0x4016c000 > > so if special alignment needs to be provided, this must happen via > kernel logic which identifies the fd as referring to /dev/db0. Exactly - because the mmap is from a device it will be the device driver which handles the mmap call and it may well choose to deliberately allocate the memory at a particular alignment. Tom -- Tom Hughes (to...@co...) http://compton.nu/ |
|
From: J W <jwd...@go...> - 2011-06-28 11:31:07
|
> Exactly - because the mmap is from a device it will be the device driver > which handles the mmap call and it may well choose to deliberately > allocate the memory at a particular alignment. In my case it is not the device driver but the (arch-arm) implementation of arch_get_unmapped_area, which requires a special alignment of 'SHMLBA' bytes in case MAP_FIXED is requested. For arm this is 4*page_size, which VG does not know about (yet). Thanks to all for helping, Cheers! Joerg |
|
From: Tom H. <to...@co...> - 2011-06-28 12:02:44
|
On 28/06/11 12:30, J W wrote: >> Exactly - because the mmap is from a device it will be the device driver >> which handles the mmap call and it may well choose to deliberately >> allocate the memory at a particular alignment. > > In my case it is not the device driver but the (arch-arm) > implementation of arch_get_unmapped_area, which requires a special > alignment of 'SHMLBA' bytes in case MAP_FIXED is requested. > For arm this is 4*page_size, which VG does not know about (yet). So you're saying that any call with MAP_FIXED on arm needs to use an address that is aligned on a 4*page boundary? If that was true I would expect valgrind to never work on arm as it pretty much always does MAP_FIXED maps, even when the program didn't actually ask for a fixed map. [ ... goes away and reads arch/arm/mm/mmap.c ... ] So it looks like that is only true for MAP_SHARED mappings, and only when the CPU's cache meets certain criteria. It is also specific to ARMv6 as the code to test the cache type is only compiled on ARMv6 and on other versions just assumes the cache will not be of the type that causes the problem. As valgrind only officially supports ARMv7 and later we shouldn't ever encounter that case on a supported platform. Tom -- Tom Hughes (to...@co...) http://compton.nu/ |
|
From: J W <jwd...@go...> - 2011-06-28 14:38:39
|
> As valgrind only officially supports ARMv7 and later we shouldn't ever > encounter that case on a supported platform. Sad but true. So I will just add some additional alignment into my "poor v6 user"-patch then... Joerg |
|
From: John R. <jr...@bi...> - 2011-06-28 15:20:27
|
On 06/28/2011 05:02 AM, Tom Hughes wrote:
> On 28/06/11 12:30, J W wrote:
>> In my case it is not the device driver but the (arch-arm)
>> implementation of arch_get_unmapped_area, which requires a special
>> alignment of 'SHMLBA' bytes in case MAP_FIXED is requested.
>> For arm this is 4*page_size, which VG does not know about (yet).
> So you're saying that any call with MAP_FIXED on arm needs to use an
> address that is aligned on a 4*page boundary?
No, only for mmap() of some hardware devices, and only because ARMv6 cache
needs it, as Tom remarks later.
If (NULL==addr && MAP_SHARED && !MAP_FIXED) {possibly && PROT_WRITE}
were considered by valgrind on ARM as an implicit request for "align to 4*page"
then J W would be happy, and the possible slight increase in fragmentation
due to general increase in alignment probably would not bother too many users.
After a couple years this concession to ARMv6 could be dropped from valgrind.
--
|
|
From: J W <jwd...@go...> - 2011-06-30 19:13:15
|
Actually it seems that "stock" VG is taking care of the additional
alignment requirements of (some?) arm platforms already on other
places, e.g (syswrap-generic.c).
if (arg1 == 0) {
/* arm-linux only: work around the fact that
VG_(am_get_advisory_client_simple) produces something that is
VKI_PAGE_SIZE aligned, whereas what we want is something
VKI_SHMLBA aligned, and VKI_SHMLBA >= VKI_PAGE_SIZE. Hence
increase the request size by VKI_SHMLBA - VKI_PAGE_SIZE and
then round the result up to the next VKI_SHMLBA boundary.
See bug 222545 comment 15. So far, arm-linux is the only
platform where this is known to be necessary. */
vg_assert(VKI_SHMLBA >= VKI_PAGE_SIZE);
if (VKI_SHMLBA > VKI_PAGE_SIZE) {
segmentSize += VKI_SHMLBA - VKI_PAGE_SIZE;
}
Given that, is it still considered as too much work for old
architectures putting sth. similar into the mmap wrapper?
Joerg
|
|
From: J W <jwd...@go...> - 2011-06-30 19:29:58
|
Adding those few lines around in "am_get_advisory" in
ML_(generic_PRE_sys_mmap) resolves the issue I have.
+ /* handle alignment to 4 pages we need for MAP_FIXED to succeed on ARM */
+ vg_assert(VKI_SHMLBA >= VKI_PAGE_SIZE);
+ if ( (arg4 & VKI_MAP_SHARED) && (arg1 == 0) && (VKI_SHMLBA >
VKI_PAGE_SIZE) ) {
+ mreq.len += VKI_SHMLBA - VKI_PAGE_SIZE;
+ }
/* Enquire ... */
advised = VG_(am_get_advisory)( &mreq, True/*client*/, &mreq_ok );
+ if (mreq_ok && (arg4 & VKI_MAP_SHARED) && (arg1 == 0) &&
(VKI_SHMLBA > VKI_PAGE_SIZE) ) {
+ Addr newaddr = VG_ROUNDUP(advised, VKI_SHMLBA);
+ mreq.len -= (newaddr - advised);
+ advised = newaddr;
+ }
|
|
From: Julian S. <js...@ac...> - 2011-07-01 07:02:01
|
J W, seems like you have quite a few patches for armv6 support now.
Can you open a bug report and attach those patches to it?
J
On Thursday, June 30, 2011, J W wrote:
> Adding those few lines around in "am_get_advisory" in
> ML_(generic_PRE_sys_mmap) resolves the issue I have.
>
> + /* handle alignment to 4 pages we need for MAP_FIXED to succeed on ARM
> */ + vg_assert(VKI_SHMLBA >= VKI_PAGE_SIZE);
> + if ( (arg4 & VKI_MAP_SHARED) && (arg1 == 0) && (VKI_SHMLBA >
> VKI_PAGE_SIZE) ) {
> + mreq.len += VKI_SHMLBA - VKI_PAGE_SIZE;
> + }
> /* Enquire ... */
> advised = VG_(am_get_advisory)( &mreq, True/*client*/, &mreq_ok );
> + if (mreq_ok && (arg4 & VKI_MAP_SHARED) && (arg1 == 0) &&
> (VKI_SHMLBA > VKI_PAGE_SIZE) ) {
> + Addr newaddr = VG_ROUNDUP(advised, VKI_SHMLBA);
> + mreq.len -= (newaddr - advised);
> + advised = newaddr;
> + }
>
> ---------------------------------------------------------------------------
> --- All of the data generated in your IT infrastructure is seriously
> valuable. Why? It contains a definitive record of application performance,
> security threats, fraudulent activity, and more. Splunk takes this data
> and makes sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> _______________________________________________
> Valgrind-developers mailing list
> Val...@li...
> https://lists.sourceforge.net/lists/listinfo/valgrind-developers
|
|
From: J W <jwd...@go...> - 2011-07-01 13:06:37
|
Yes, will do asap. Thanks! |