#48 Use Windows virtual memory where possible

open
nobody
None
5
2008-02-28
2008-02-28
No

coLinux provides a setting how much memory to use.

As far as I understand the memory is allocated by the driver linux.sys and located at fixed addresses as shared memory.

That is (maybe) required for the way how coLinux communicates with Windows. (Discussing the communication is a different topic).

But it seems that also all Linux applications running under coLinux are using that fixed shared memory too.

What about giving normal virtual memory to Linux applications?
-> This would elminate the memory setting, since only the small amount required for communication needs to be fixed shared memory.
-> Further no need to setup a Linux swap file/partition since this is also covered by Windows.

Well, just a thought to easy configuration and usablity.

Bebbo

Discussion

  • Brian

    Brian - 2008-02-28

    Logged In: YES
    user_id=585719
    Originator: NO

    Doesn't this already happen implicilty as a result of disk cacheing on the Windows side? For example, say you set the memory size to 128MB and create a 1GB swapfile. When the colinux guest needs to swap, that will result in regular I/O on the windows side to the file containing the swap area, which should be covered by the normal Windows cache manager like all file access.

     
  • Stefan "Bebbo" Franke

    Logged In: YES
    user_id=406985
    Originator: YES

    The Windows cache manager might help somehow, but:

    - allocating fixed shared memory has the same effect as having less physical memory. Using mem=512 means your Windows has 512MB less memory available. Thus Windows might swap more often
    - 2 separate swap mechanism with partial memory (e.g. 512MB and 1.5 GB) are always less efficient than one swap mechanism with the full memory (e.g. 2GB).
    - coLinux file I/O is always piped through to windows and thus slower than direct Windows I/O

    => there are many benefits of using the host as direct as possible wherever capable.

    Bebbo

     
  • Henry N.

    Henry N. - 2008-02-28

    Logged In: YES
    user_id=579204
    Originator: NO

    Best result you would have with a swapfile (not swap partion) on windows side. If Linux would swap some memory, the Windows host becomes a write to file, and this is cached (write cache). So, such swap out from Linux is not all times a write to hardware hard disk. The real swapping to hard disk begins, if Windows has not enouth memory to hold the swapfile in cache.

    That's why the swapfile on windows side should no bigger as 2*total RAM.

    Using Windows Virtual memory is not doable. - Currently not. Windows can swapout Virtual memory any time. And if such page is in use Linux side, the Linux kernel must inform about. This is a problem: Linux holds self the page-use-counters and give the page free if the no longer need. Linux and Windows don't know about the other side OS.

    The memory is not fixed since coLinux 0.7.x. If Linux give the pseudo-physicaly memory free, then coLinux driver give the memory also free for the Windows side. The Linux memory is not allocated at boot time, (VMware does it, I'm afraid)

    The current implementation lets Linux kernel do his thing with memory up to the "mem" limit you was set. That makes less OS swichers we can have. OS switches cost time. And an OS switch for getting a virtual page from "free memory" area is not good idea.

    If would implement your suggestion, then the Linux kernel would not have any memory managment routines. I mean routines, that say whenever a page can free out to swap and when needs to allocate more pages, and what pages can use for file buffers...

     
  • Stefan "Bebbo" Franke

    Logged In: YES
    user_id=406985
    Originator: YES

    > If would implement your suggestion, then the Linux kernel would not have
    > any memory managment routines. I mean routines, that say whenever a page
    > can free out to swap and when needs to allocate more pages, and what pages
    > can use for file buffers...

    Exactly! Since everything is already covered by Windows there is no need for further memory management on the Linux side.

    It will even work for file buffers:
    a) if the Linux file buffers are used frequently Windows wont swap that memory out to disk
    b) if the Linux file buffers are swapped out by Windows, they will be sapped in and then be used.

    Since coLinux accesses no hardware directly every access goes via Windows and thus uses Windows memory, caches and so on.

    Since omitting is easier than implementing I prefer to omit redundant implementations.

     
  • Nobody/Anonymous

    Logged In: NO

    There is another important advantage: if colinux will use directly the virtual memory every application that runs under linux can access memory initialized by windows apis that means that for example cofb can work using an overlay initialized using SDL/DX/GL with high performance

    cofb linux kernel module can ask to the system to initiaialize an overlay and recive as response the location, in the virtual memory, of the overlay!

     
  • Nobody/Anonymous

    Logged In: NO

    <quote>
    There is another important advantage: if colinux will use directly the
    virtual memory every application that runs under linux can access memory
    initialized by windows apis that means that for example cofb can work using
    an overlay initialized using SDL/DX/GL with high performance
    </quote>

    Not so much. Those are user-mode APIs and the memory would be automatically released when the allocating process terminates. If colinux was using the memory it would fail, and because colinux is a driver, it could bring down the whole system.

    colinux is already capable of mapping windows memory regions into the linux virtual memory map, but those regions need to be allocated in kernel mode so they aren't freed prematurely.

     
  • Henry N.

    Henry N. - 2008-03-30

    Logged In: YES
    user_id=579204
    Originator: NO

    Hi,

    <quote>
    colinux is already capable of mapping windows memory regions into the
    linux virtual memory map, but those regions need to be allocated in kernel
    mode so they aren't freed prematurely.
    </quote>

    Is not all right. Linux kernel difer between "hot" and "cold" pages. Hot pages are not freeing on host side. But cold pages will be freeing also for the host. You can see it in the Windows Taskmanager, that the pool of physically memory goes up and down after freeing memory inside coLinux guest.

    Last days found a nice side effect for this discussion.
    Using Windows virtual memory menagment cost many cpu time!

    In the last snapshot-20080329 http://www.colinux.org/snapshots/ have removed the call to Windows host and asking for "new" memory page. The consents is, that coLinux runs 50% faster now. Applications without disk-io runs in 30% of old time (90s before, 30s now).

     
  • Nobody/Anonymous

    Logged In: NO

    <quote>
    Is not all right. Linux kernel difer between "hot" and "cold" pages. Hot
    pages are not freeing on host side. But cold pages will be freeing also for
    the host.
    </quote>

    But these "cold" pages are freed by colinux itself, correct? For colinux to map memory owned by some other process which the other process later freed would be disastrous.

     
  • Nobody/Anonymous

    Logged In: NO

    Yes, of curse colinux controls all the map and unmap.
    But in the case of cold and hot pages we talk about pages, that are currently not mapped in Linux. These pages are in the list of free pages, and no process needs this pages.

     
  • Henry N.

    Henry N. - 2008-03-31

    Logged In: YES
    user_id=579204
    Originator: NO

    Last post was from me. Sorry, was not logged in.

    Henry

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks