AspaceManager

Changes to the address space manager

The following information applies to virtual memory management on 32 bit Windows, including the WOW64 subsystem on 64 bit Windows.

Allocation Granularity

While both Posix and Windows use the same page size of 4 KB, Windows allocates virtual memory at addresses that are a multiple of "allocation granularity", not page size. Allocation granularity can be retrieved with the "GetSystemInfo" call (struct SYSTEM_INFO member dwAllocationGranularity), and is usually 64 KB.

http://blogs.msdn.com/b/oldnewthing/archive/2003/10/08/55239.aspx

Valgrind has to take that policy into account when allocating virtual memory. VG_(am_get_advisory) has been changed to only return start addresses that are granularity aligned.

aspacem_minAddr, aspacem_maxAddr, aspacem_cStart, aspacem_vStart and suggested_clstack_top all are asserted to be granularity aligned.

Memory layout WIN32 & WOW64

Windows loads user and system DLLs starting at 0x70000000. Adresses above 0x80000000 (2 GB) are inaccessible, so a process has only 2 GB not 4 GB. The lower 64 MB contain lots of allocated pages by Windows kernel or libraries. The program standard load address is 0x00400000 (4 MB). Valgrind tools load at 0x38000000 (896 MB). The strategy is to use the address space between 0x04000000 and 0x7000000, dividing it equally between the client and Valgrind.

System:   0x00000000 - 0x04000000   (   0 MB -   64 MB)

Client:   0x04000000 - 0x39ff0000   (  64 MB -  927 MB)

Valgrind: 0x39ff0000 - 0x70000000   ( 927 MB - 1792 MB)

DLLs:     0x70000000 - 0x80000000   (1792 MB - 2048 MB)

Virtual Memory changes by Syscalls

Various syscalls and even user-space library routines allocate new regions and even change existing regions at will. A new function VG_(win_am_sync_ensure) has been added to aspacemanager that checks if there is a mismatch (using existing VG_(am_do_sync_check) functionality), and if a mismatch is found fix it by re-adding offending regions in refresh_mapping_callback() using add_segment().

Filename case insensitivity

Filenames are case-insensitive on Windows and are used as segment names. Therefore, aspacemanager has to compare segment names in a case-insensitive way.

Redesign by Jiri

The reason why there are so many address space layout mismatches is that the address space gets changed during its inspection, which creates endless futile attempts trying to get the snapshot right.

The culprit here was retrieving name of the mapped memory areas, which used local V heap for file handle (fstat) and virtually unlimited amount of process heap (conversion from "ANSI" to Unicode in xxxA API, conversion from DOS path to NT path, relative path names etc.).

In order to avoid these heap changes during the enumeration, I have created a new function VG_(win_os_get_mapping_filename), which does the job all at once, without creating a temporary virtual fd, and uses low-level NT calls only.

This got rid of all "sync check failed" messages for the example programs for me, so I even went ahead and started to remove some of the sync check macros. (There are still some "fixing segment mismatch" log entries, which are OK and can be solved later - they are just from manual updates after blocks of code known to possibly change memory layout.)

This helper function cleaned some places where mapped file information need to be retrieved and VG_(win_os_replace_device_with_driveletter) is no longer needed externally anymore and was converted into just a local function.

I have also converted as much API calls as possible to use the xxxW Unicode versions to avoid the unnecessary buffer allocation by the kernel. For this, I decided to go with UTF-8 as internal encoding for filenames etc. It will allow us to work with international paths, but stays compatible with the rest of Valgrind passing char* around and UTF-8 is the encoding used on Linux etc. too anyway.