basilisk-devel

[B2-devel] Macintosh memory information?

[Alexander v. G. <kal...@un...>]

Does anyone have any documentation on the memory layout of the old world 
Macintosh? (Performa 6400 era)

I've been looking to rework how SheepShear manages memory and it's not 
completely clear to me how it does
it's memory translation.  Most of the bugs I've seen on x86_64 are due to 
the exact memory mapping on emulated
powerpc systems (eg, hard coded memory addresses we *have* to map for things 
to function) and something getting mangled
on x86_64.
https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089

I see memory maps for the really early Mac's, but am failing to find 
anything for the later old world machines
online or in the old Macintosh service manuals.

Here is what I know for a fact looking at the code:

Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
RAM,               0x0        - (memory size selected less then 1GB)
ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1 
host:guest location)

Real addressing == "normal" allocation where memory locations are dynamic, 
and we translate addresses to match
Direct addressing == memory is directly addressed, mostly used on native 
PowerPC?
Nat_mem offset == ???, guessing it's for native PowerPC

It looks given the code below that the Macintosh has to be patched to use 
the host memory
addresses vs the emulator doing the memory translation (SheepShear acting as 
a mmu to the guest)
https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185

The reason I ask here is because these changes could be pulled upstream to 
SheepShaver.

  -- Thanks!
    Alex

[B2-devel] Macintosh memory information?

[Alexander v. G. <kal...@un...>]

Does anyone have any documentation on the memory layout of the old world 
Macintosh? (Performa 6400 era)

I've been looking to rework how SheepShear manages memory and it's not 
completely clear to me how it does
it's memory translation.  Most of the bugs I've seen on x86_64 are due to 
the exact memory mapping on emulated
powerpc systems (eg, hard coded memory addresses we *have* to map for things 
to function) and something getting mangled
on x86_64.
https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089

I see memory maps for the really early Mac's, but am failing to find 
anything for the later old world machines
online or in the old Macintosh service manuals.

Here is what I know for a fact looking at the code:

Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
RAM,               0x0        - (memory size selected less then 1GB)
ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1 
host:guest location)

Real addressing == "normal" allocation where memory locations are dynamic, 
and we translate addresses to match
Direct addressing == memory is directly addressed, mostly used on native 
PowerPC?
Nat_mem offset == ???, guessing it's for native PowerPC

It looks given the code below that the Macintosh has to be patched to use 
the host memory
addresses vs the emulator doing the memory translation (SheepShear acting as 
a mmu to the guest)
https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185

The reason I ask here is because these changes could be pulled upstream to 
SheepShaver.

  -- Thanks!
    Alex

Re: [B2-devel] Macintosh memory information?

[Frank T. <fra...@gm...>]

I think that you need the PowerPC edition/addition of Inside Macintosh. I
will try to check on this in a few hours. I believe that there is a copy
available on-line somewhere, but I can scan the pages that you need if not.

On Wed, Jul 18, 2012 at 7:21 AM, Alexander von Gluck
<kal...@un...>wrote:

> Does anyone have any documentation on the memory layout of the old world
> Macintosh? (Performa 6400 era)
>
> I've been looking to rework how SheepShear manages memory and it's not
> completely clear to me how it does
> it's memory translation.  Most of the bugs I've seen on x86_64 are due to
> the exact memory mapping on emulated
> powerpc systems (eg, hard coded memory addresses we *have* to map for
> things
> to function) and something getting mangled
> on x86_64.
> https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089
>
> I see memory maps for the really early Mac's, but am failing to find
> anything for the later old world machines
> online or in the old Macintosh service manuals.
>
> Here is what I know for a fact looking at the code:
>
> Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
> RAM,               0x0        - (memory size selected less then 1GB)
> ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1
> host:guest location)
>
> Real addressing == "normal" allocation where memory locations are dynamic,
> and we translate addresses to match
> Direct addressing == memory is directly addressed, mostly used on native
> PowerPC?
> Nat_mem offset == ???, guessing it's for native PowerPC
>
> It looks given the code below that the Macintosh has to be patched to use
> the host memory
> addresses vs the emulator doing the memory translation (SheepShear acting
> as
> a mmu to the guest)
>
> https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185
>
> The reason I ask here is because these changes could be pulled upstream to
> SheepShaver.
>
>   -- Thanks!
>     Alex
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> basilisk-devel mailing list
> bas...@li...
> https://lists.sourceforge.net/lists/listinfo/basilisk-devel
>

Re: [B2-devel] Macintosh memory information?

[Frank T. <fra...@gm...>]

There is stuff of interest in Inside Macintosh: PowerPC System Software and
Inside Macintosh: Memory. The former is available on-line but not the
latter. This (
http://www.mindfiresolutions.com/mindfire/Mac_Memory_Manager.pdf ) may have
some of what you need.

On Wed, Jul 18, 2012 at 8:49 AM, Frank Trampe <fra...@gm...>wrote:

> I think that you need the PowerPC edition/addition of Inside Macintosh. I
> will try to check on this in a few hours. I believe that there is a copy
> available on-line somewhere, but I can scan the pages that you need if not.
>
>
> On Wed, Jul 18, 2012 at 7:21 AM, Alexander von Gluck <
> kal...@un...> wrote:
>
>> Does anyone have any documentation on the memory layout of the old world
>> Macintosh? (Performa 6400 era)
>>
>> I've been looking to rework how SheepShear manages memory and it's not
>> completely clear to me how it does
>> it's memory translation.  Most of the bugs I've seen on x86_64 are due to
>> the exact memory mapping on emulated
>> powerpc systems (eg, hard coded memory addresses we *have* to map for
>> things
>> to function) and something getting mangled
>> on x86_64.
>> https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089
>>
>> I see memory maps for the really early Mac's, but am failing to find
>> anything for the later old world machines
>> online or in the old Macintosh service manuals.
>>
>> Here is what I know for a fact looking at the code:
>>
>> Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
>> RAM,               0x0        - (memory size selected less then 1GB)
>> ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1
>> host:guest location)
>>
>> Real addressing == "normal" allocation where memory locations are dynamic,
>> and we translate addresses to match
>> Direct addressing == memory is directly addressed, mostly used on native
>> PowerPC?
>> Nat_mem offset == ???, guessing it's for native PowerPC
>>
>> It looks given the code below that the Macintosh has to be patched to use
>> the host memory
>> addresses vs the emulator doing the memory translation (SheepShear acting
>> as
>> a mmu to the guest)
>>
>> https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185
>>
>> The reason I ask here is because these changes could be pulled upstream to
>> SheepShaver.
>>
>>   -- Thanks!
>>     Alex
>>
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> basilisk-devel mailing list
>> bas...@li...
>> https://lists.sourceforge.net/lists/listinfo/basilisk-devel
>>
>
>

Re: [B2-devel] Macintosh memory information?

[Alexander v. G. <kal...@un...>]

All great stuff, definitely helps! :)

Thanks!
  -- Alex

On 18.07.2012 09:29, Frank Trampe wrote:
> There is stuff of interest in Inside Macintosh: PowerPC System Software 
> and Inside Macintosh: Memory. The former is
> available on-line but not the latter. This ( 
> http://www.mindfiresolutions.com/mindfire/Mac_Memory_Manager.pdf [7] )
> may have some of what you need.
>
> On Wed, Jul 18, 2012 at 8:49 AM, Frank Trampe <fra...@gm... [8]> 
> wrote:
>
>> I think that you need the PowerPC edition/addition of Inside Macintosh. I 
>> will try to check on this in a few hours.
>> I believe that there is a copy available on-line somewhere, but I can 
>> scan the pages that you need if not.
>>
>> On Wed, Jul 18, 2012 at 7:21 AM, Alexander von Gluck 
>> <kal...@un... [6]> wrote:
>>
>>> Does anyone have any documentation on the memory layout of the old world
>>> Macintosh? (Performa 6400 era)
>>>
>>> I've been looking to rework how SheepShear manages memory and it's not
>>> completely clear to me how it does
>>> it's memory translation.  Most of the bugs I've seen on x86_64 are due 
>>> to
>>> the exact memory mapping on emulated
>>> powerpc systems (eg, hard coded memory addresses we *have* to map for 
>>> things
>>> to function) and something getting mangled
>>> on x86_64.
>>> https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089 
>>> [1]
>>>
>>> I see memory maps for the really early Mac's, but am failing to find
>>> anything for the later old world machines
>>> online or in the old Macintosh service manuals.
>>>
>>> Here is what I know for a fact looking at the code:
>>>
>>> Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
>>> RAM,               0x0        - (memory size selected less then 1GB)
>>> ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1
>>> host:guest location)
>>>
>>> Real addressing == "normal" allocation where memory locations are 
>>> dynamic,
>>> and we translate addresses to match
>>> Direct addressing == memory is directly addressed, mostly used on native
>>> PowerPC?
>>> Nat_mem offset == ???, guessing it's for native PowerPC
>>>
>>> It looks given the code below that the Macintosh has to be patched to 
>>> use
>>> the host memory
>>> addresses vs the emulator doing the memory translation (SheepShear 
>>> acting as
>>> a mmu to the guest)
>>> 
>>> https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185 
>>> [2]
>>>
>>> The reason I ask here is because these changes could be pulled upstream 
>>> to
>>> SheepShaver.
>>>
>>>   -- Thanks!
>>>     Alex
>>>
>>> 
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. 
>>> Discussions
>>> will include endpoint security, mobile security and the latest in 
>>> malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ [3]
>>> _______________________________________________
>>> basilisk-devel mailing list
>>> bas...@li... [4]
>>> https://lists.sourceforge.net/lists/listinfo/basilisk-devel [5]
>
>
>
> Links:
> ------
> [1] https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089
> [2] 
> https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185
> [3] http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> [4] mailto:bas...@li...
> [5] https://lists.sourceforge.net/lists/listinfo/basilisk-devel
> [6] mailto:kal...@un...
> [7] http://www.mindfiresolutions.com/mindfire/Mac_Memory_Manager.pdf
> [8] mailto:fra...@gm...

Re: [B2-devel] Macintosh memory information?

[Michael S. <msb...@me...>]

Inside Macintosh: Memory is available at <http://developer.apple.com/legacy/mac/library/#documentation/mac/Memory/Memory-2.html>


As far as I can tell (and I could be wrong)...

Basilisk II supports three types of memory access modes. Which one to use is selected at compile time:

REAL: The addresses seen inside the emulated machine are exactly the same as the host memory address.

DIRECT: The addresses are shifted by a fixed offset. It calculates it at runtime, by allocating the guest memory block anywhere the host can find space, and then calculating the offset between the guest and host addresses.

VIRTUAL:  The guest address space is broken down into banks, e.g. the RAM, ROM and frame buffer are handled separately. It uses some kind of address translation logic.


SheepShaver (at least the 32-bit version, I don't know about 64-bit) supports REAL and DIRECT addressing modes, but not VIRTUAL.

I don't know if anyone uses DIRECT in SheepShaver, because it has a fatal flaw.  Unlike Basilisk II, the offset is calculated at *compile* time.  But this only works if each user compiles the code!  And if something changes the library address randomization (such as applying an upgrade) it invalidates the compiled-in offset. You would need to keep recompiling it. Of course this is completely non-portable.

That leaves REAL mode.

When running on PPC, REAL mode allows it to run virtualized instead of emulated, where it can just let the native processor loose on the code.  Running on Intel there shouldn't be a penalty for doing an address shift, since it has to emulate the CPU and anyway. But it doesn't shift it, due to the way DIRECT is implemented as described above.


OK, given that, what about restrictions?

Before I messed with the memory allocation, SheepShaver's host memory map was:


  RAM area       ???M   0x20000000 to ? 
  ROM area:        5M   0x40800000 to 0x40D00000 
  Kernel Data 2:   8K   0x5fffe000 to 0x60000000 
  SS Globals:    512K   0x60000000 to 0x60080000 
  Sig stack:      64K   0x60080000 to 0x60090000 
  DR Emulator:    64K   0x68070000 to 0x68080000 
  Kernal Data 1:   8K   0x68ffe000 to 0x69000000 
  DR Cache:      512K   0x69000000 to 0x69080000 
  Load mod:             0x78048000 


The system globals have to be at guest address 0.  SheepShaver uses several techniques to achieve this, including attempting to allocate the RAM block at 0, using linking magic to get a program segment to load at 0, or direct allocation of a small amount of storage at 0. (Note that allocating at zero is problematic in Linux, because the default setting in some Linux versions is to not allow it.)

Mac OS doesn't care where the RAM or ROM blocks start. The starting addresses are just plugged into a global.

However, I did observe that:

1. It crashes if the guest ROM address isn't on a 1 MB boundary. 

I'm not sure if that is a Mac OS restriction or something in SheepShaver.  I suspect this code in rsrc_patches.cpp has something to do with it:

	// Patch native GetResource()
	uint32 upp = ReadMacInt32(0x1480);
	if ((upp & 0xffc00000) == ROMBase)
		return;

2. It crashes if the guest ROM address is not higher than the RAM address.

I worked around this by allocating the RAM and ROM together as one block, with enough extra space to permit the ROM to be aligned.

This ensures that the ROM is higher than RAM, but at a disadvantage: it increases the size of the memory block, which makes it more likely that a) it might not get it, or b) it may get it but run into another problem...


3. It crashes if the RAM is allocated too high.

Higher than what? It must be one of the "other areas" (Kernel Data, Sig stack, etc.), but which one?  Is it a Mac OS restriction or something in SheepShaver?

I worked around it by checking if it did load higher than the kernel data address, and if so it just gives up. This obviously is not an ideal situation.


So, per your question, SheepShaver on Mac and Unix no longer load the ROM to be at 0x40800000, and it only "tries" to load RAM at 0x0.  It is more likely that it loads it higher.


Windows is differnet. Since it didn't have the problem finding memory (since its address space wasn't polluted by Cocoa), there was no reason to change SheepShaver to shift memory.



On Jul 18, 2012, at 7:20 AM, Alexander von Gluck wrote:

> Does anyone have any documentation on the memory layout of the old world 
> Macintosh? (Performa 6400 era)
> 
> I've been looking to rework how SheepShear manages memory and it's not 
> completely clear to me how it does
> it's memory translation.  Most of the bugs I've seen on x86_64 are due to 
> the exact memory mapping on emulated
> powerpc systems (eg, hard coded memory addresses we *have* to map for things 
> to function) and something getting mangled
> on x86_64.
> https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089
> 
> I see memory maps for the really early Mac's, but am failing to find 
> anything for the later old world machines
> online or in the old Macintosh service manuals.
> 
> Here is what I know for a fact looking at the code:
> 
> Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
> RAM,               0x0        - (memory size selected less then 1GB)
> ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1 
> host:guest location)
> 
> Real addressing == "normal" allocation where memory locations are dynamic, 
> and we translate addresses to match
> Direct addressing == memory is directly addressed, mostly used on native 
> PowerPC?
> Nat_mem offset == ???, guessing it's for native PowerPC
> 
> It looks given the code below that the Macintosh has to be patched to use 
> the host memory
> addresses vs the emulator doing the memory translation (SheepShear acting as 
> a mmu to the guest)
> https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185
> 
> The reason I ask here is because these changes could be pulled upstream to 
> SheepShaver.
> 
> -- Thanks!
>  Alex
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> basilisk-devel mailing list
> bas...@li...
> https://lists.sourceforge.net/lists/listinfo/basilisk-devel

Re: [B2-devel] Macintosh memory information?

[Alexander v. G. <kal...@un...>]

On 18.07.2012 23:01, Michael Schmitt wrote:
> Inside Macintosh: Memory is available at
> 
> <http://developer.apple.com/legacy/mac/library/#documentation/mac/Memory/Memory-2.html>
>
> As far as I can tell (and I could be wrong)...
>
> Basilisk II supports three types of memory access modes. Which one to use 
> is selected at compile time:
>
> REAL: The addresses seen inside the emulated machine are exactly the same 
> as the host memory address.
>
> DIRECT: The addresses are shifted by a fixed offset. It calculates it at 
> runtime, by allocating the guest memory
> block anywhere the host can find space, and then calculating the offset 
> between the guest and host addresses.
>
> VIRTUAL:  The guest address space is broken down into banks, e.g. the RAM, 
> ROM and frame buffer are handled
> separately. It uses some kind of address translation logic.

These definitions seem to be swapped (as far as I can tell)
https://github.com/kallisti5/sheepshear/blob/master/src/platform/Unix/main_unix.cpp#L944

The UNIX code above seems to allocate the memory anywhere available with 
REAL addressing, and at a fixed location
with DIRECT addressing.

I haven't seen and references to VIRTUAL memory access mode though.

> SheepShaver (at least the 32-bit version, I don't know about 64-bit) 
> supports REAL and DIRECT addressing modes, but
> not VIRTUAL.
>
> I don't know if anyone uses DIRECT in SheepShaver, because it has a fatal 
> flaw.  Unlike Basilisk II, the offset is
> calculated at *compile* time.  But this only works if each user compiles 
> the code!  And if something changes the
> library address randomization (such as applying an upgrade) it invalidates 
> the compiled-in offset. You would need to
> keep recompiling it. Of course this is completely non-portable.

This may explain some of the large emulation problems seen on newer gcc + 
x86_64

> That leaves REAL mode.
>
> When running on PPC, REAL mode allows it to run virtualized instead of 
> emulated, where it can just let the native
> processor loose on the code.  Running on Intel there shouldn't be a 
> penalty for doing an address shift, since it has
> to emulate the CPU and anyway. But it doesn't shift it, due to the way 
> DIRECT is implemented as described above.


Given these limitations, can we put a wrapper around memory calls and adjust 
the memory offsets at run time? (enabling
us to cherrypick whichever host memory we want to? (but we should make sure 
it is aligned to 1MB))

Example:
   Mac2HostAddr, Host2MacAddr can do address translation, so why not do the 
following:

   1) Take a hunk of memory, allocate it large enough for RAM and ROM 
(aligned) for easy debugging
   2) Set the RAMBaseHost to the start of the chunk of memory
   3) Set the ROMBaseHost to the start of the chunk of memory + RAM size 
(which should be at least 1MB intervals)

   *) Mac2HostAddr - If we catch a RAM address (0 - 1GB) with Mac2HostAddr, 
point it to RAMBaseHost + mac offset
   *) Mac2HostAddr - If we catch a ROM address (0x40800000 - 0x40D00000), 
point it to the ROMBaseHost + mac offset
   *) Host2MacAddr - If we catch RAMBaseHost - RAM Size, subtract RAMBaseHost
   *) Host2MacAddr - If we catch ROMBaseHost - 5MB, subtract ROMBaseHost

I don't understand the JIT stuff enough yet though to see if this would work 
with it.

Thoughts?

-- Alex

>
> On Jul 18, 2012, at 7:20 AM, Alexander von Gluck wrote:
>
>> Does anyone have any documentation on the memory layout of the old world
>> Macintosh? (Performa 6400 era)
>>
>> I've been looking to rework how SheepShear manages memory and it's not
>> completely clear to me how it does
>> it's memory translation.  Most of the bugs I've seen on x86_64 are due to
>> the exact memory mapping on emulated
>> powerpc systems (eg, hard coded memory addresses we *have* to map for 
>> things
>> to function) and something getting mangled
>> on x86_64.
>> https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089
>>
>> I see memory maps for the really early Mac's, but am failing to find
>> anything for the later old world machines
>> online or in the old Macintosh service manuals.
>>
>> Here is what I know for a fact looking at the code:
>>
>> Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
>> RAM,               0x0        - (memory size selected less then 1GB)
>> ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1
>> host:guest location)
>>
>> Real addressing == "normal" allocation where memory locations are 
>> dynamic,
>> and we translate addresses to match
>> Direct addressing == memory is directly addressed, mostly used on native
>> PowerPC?
>> Nat_mem offset == ???, guessing it's for native PowerPC
>>
>> It looks given the code below that the Macintosh has to be patched to use
>> the host memory
>> addresses vs the emulator doing the memory translation (SheepShear acting 
>> as
>> a mmu to the guest)
>> 
>> https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185
>>
>> The reason I ask here is because these changes could be pulled upstream 
>> to
>> SheepShaver.
>>
>> -- Thanks!
>>  Alex
>>
>> 
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> basilisk-devel mailing list
>> bas...@li...
>> https://lists.sourceforge.net/lists/listinfo/basilisk-devel
>
>
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> basilisk-devel mailing list
> bas...@li...
> https://lists.sourceforge.net/lists/listinfo/basilisk-devel

Re: [B2-devel] Macintosh memory information?

[Michael S. <msb...@me...>]

On Jul 19, 2012, at 7:23 AM, Alexander von Gluck wrote:

> On 18.07.2012 23:01, Michael Schmitt wrote:
>> Inside Macintosh: Memory is available at
>> 
>> <http://developer.apple.com/legacy/mac/library/#documentation/mac/Memory/Memory-2.html>
>> 
>> As far as I can tell (and I could be wrong)...
>> 
>> Basilisk II supports three types of memory access modes. Which one to use 
>> is selected at compile time:
>> 
>> REAL: The addresses seen inside the emulated machine are exactly the same 
>> as the host memory address.
>> 
>> DIRECT: The addresses are shifted by a fixed offset. It calculates it at 
>> runtime, by allocating the guest memory
>> block anywhere the host can find space, and then calculating the offset 
>> between the guest and host addresses.
>> 
>> VIRTUAL:  The guest address space is broken down into banks, e.g. the RAM, 
>> ROM and frame buffer are handled
>> separately. It uses some kind of address translation logic.
> 
> These definitions seem to be swapped (as far as I can tell)
> https://github.com/kallisti5/sheepshear/blob/master/src/platform/Unix/main_unix.cpp#L944
> 
> The UNIX code above seems to allocate the memory anywhere available with 
> REAL addressing, and at a fixed location
> with DIRECT addressing.
> 
> I haven't seen and references to VIRTUAL memory access mode though.

That's because first I was describing how the Basilisk II code works, from which SheepShaver is derived.

BII has VIRTUAL, SS doesn't.

In REAL mode SS allocates the HOST block where ever it can. But once it is allocated, the address the guest sees is exactly the host address where the block was allocated.

In DIRECT mode the memory is allocated at a fixed address (that in SS is determined at compile time), and then the addresses are translated so the guest sees RAM starting at zero.


>> SheepShaver (at least the 32-bit version, I don't know about 64-bit) 
>> supports REAL and DIRECT addressing modes, but
>> not VIRTUAL.
>> 
>> I don't know if anyone uses DIRECT in SheepShaver, because it has a fatal 
>> flaw.  Unlike Basilisk II, the offset is
>> calculated at *compile* time.  But this only works if each user compiles 
>> the code!  And if something changes the
>> library address randomization (such as applying an upgrade) it invalidates 
>> the compiled-in offset. You would need to
>> keep recompiling it. Of course this is completely non-portable.
> 
> This may explain some of the large emulation problems seen on newer gcc + 
> x86_64

Does it use DIRECT mode?

I've only used SheepShaver in REAL mode.

>> That leaves REAL mode.
>> 
>> When running on PPC, REAL mode allows it to run virtualized instead of 
>> emulated, where it can just let the native
>> processor loose on the code.  Running on Intel there shouldn't be a 
>> penalty for doing an address shift, since it has
>> to emulate the CPU and anyway. But it doesn't shift it, due to the way 
>> DIRECT is implemented as described above.
> 
> 
> Given these limitations, can we put a wrapper around memory calls and adjust 
> the memory offsets at run time? (enabling
> us to cherrypick whichever host memory we want to? (but we should make sure 
> it is aligned to 1MB))

Remember it is the guest ROM rom address that needs to be aligned. It doesn't matter if the host address is aligned.

> Example:
> Mac2HostAddr, Host2MacAddr can do address translation, so why not do the 
> following:
> 
> 1) Take a hunk of memory, allocate it large enough for RAM and ROM 
> (aligned) for easy debugging
> 2) Set the RAMBaseHost to the start of the chunk of memory
> 3) Set the ROMBaseHost to the start of the chunk of memory + RAM size 
> (which should be at least 1MB intervals)
> 
> *) Mac2HostAddr - If we catch a RAM address (0 - 1GB) with Mac2HostAddr, 
> point it to RAMBaseHost + mac offset
> *) Mac2HostAddr - If we catch a ROM address (0x40800000 - 0x40D00000), 
> point it to the ROMBaseHost + mac offset
> *) Host2MacAddr - If we catch RAMBaseHost - RAM Size, subtract RAMBaseHost
> *) Host2MacAddr - If we catch ROMBaseHost - 5MB, subtract ROMBaseHost
> 
> I don't understand the JIT stuff enough yet though to see if this would work 
> with it.
> 
> Thoughts?

Essentially that would be changing SheepShaver to use the same DIRECT mode as Basilisk II (if all addresses are shifted by the same offset), or Basilisk II's VIRTUAL mode, which currently is not in SheepShaver.

But since the Mac OS doesn't need to have the RAM and ROM at any particular address, simply shifting the host-to-guest RAM/ROM addresses doesn't buy us much.

What we would want is to translate all of the addresses: the low system globals at 0x0, the special data areas, etc.

When I looked at the DIRECT code I wasn't sure if it could handle 32-bit wrap-around access properly. Consider that if guest 0x0 is actually in the middle of the host space, then some of the high guest space is actually less than it.  What happens when a guest memory operation needs to wrap from high host to low host?

And there could be cases where SheepShaver is making an assumption that the distance between two guest addresses is the same as the distance between the two corresponding host addresses, and vice versa.


In 64-bit mode we should have huge amounts of host address space.  Can we map the entire guest address space, all 4 GB of it or whatever the classic Mac OS had, high up in 64-bit land and then shift everything -- including the 0x0 globals? Or do we still have the problem of Cocoa allocating stuff in our address space at initialization?


> -- Alex
> 
>> 
>> On Jul 18, 2012, at 7:20 AM, Alexander von Gluck wrote:
>> 
>>> Does anyone have any documentation on the memory layout of the old world
>>> Macintosh? (Performa 6400 era)
>>> 
>>> I've been looking to rework how SheepShear manages memory and it's not
>>> completely clear to me how it does
>>> it's memory translation.  Most of the bugs I've seen on x86_64 are due to
>>> the exact memory mapping on emulated
>>> powerpc systems (eg, hard coded memory addresses we *have* to map for 
>>> things
>>> to function) and something getting mangled
>>> on x86_64.
>>> https://github.com/kallisti5/sheepshear/issues/15#issuecomment-7045089
>>> 
>>> I see memory maps for the really early Mac's, but am failing to find
>>> anything for the later old world machines
>>> online or in the old Macintosh service manuals.
>>> 
>>> Here is what I know for a fact looking at the code:
>>> 
>>> Low memory area, - 0x0        - 0x3000  (static 1:1 host:guest location)
>>> RAM,               0x0        - (memory size selected less then 1GB)
>>> ROM,               0x40800000 - 0x40C00000 (4MB mac rom)  (static 1:1
>>> host:guest location)
>>> 
>>> Real addressing == "normal" allocation where memory locations are 
>>> dynamic,
>>> and we translate addresses to match
>>> Direct addressing == memory is directly addressed, mostly used on native
>>> PowerPC?
>>> Nat_mem offset == ???, guessing it's for native PowerPC
>>> 
>>> It looks given the code below that the Macintosh has to be patched to use
>>> the host memory
>>> addresses vs the emulator doing the memory translation (SheepShear acting 
>>> as
>>> a mmu to the guest)
>>> 
>>> https://github.com/kallisti5/sheepshear/blob/master/src/kpx_cpu/src/cpu/vm.hpp#L185
>>> 
>>> The reason I ask here is because these changes could be pulled upstream 
>>> to
>>> SheepShaver.
>>> 
>>> -- Thanks!
>>> Alex
>>> 
>>> 
>>> ------------------------------------------------------------------------------
>>> Live Security Virtual Conference
>>> Exclusive live event will cover all the ways today's security and
>>> threat landscape has changed and how IT managers can respond. Discussions
>>> will include endpoint security, mobile security and the latest in malware
>>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>>> _______________________________________________
>>> basilisk-devel mailing list
>>> bas...@li...
>>> https://lists.sourceforge.net/lists/listinfo/basilisk-devel
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> basilisk-devel mailing list
>> bas...@li...
>> https://lists.sourceforge.net/lists/listinfo/basilisk-devel
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> basilisk-devel mailing list
> bas...@li...
> https://lists.sourceforge.net/lists/listinfo/basilisk-devel