SystemCalls

System Calls

This page describes how system calls on different Windows systems are performed.

• WIN32 : 32 bit process on a 32 bit Windows
• WOW64 : 32 bit process on a 64 bit Windows
• WIN64 : 64 bit process on a 64 bit Windows

WIN32

• caller calls system call stub
• system call stub calls ntdll!KiFastSystemCall
• ntdll!KiFastSystemCall enters the kernel via SYSENTER
• nnnnnnnn = system call number
• ssssssss = address of !SharedUserData!SystemCallStub, which contains address of ntdll!KiFastSystemCall
• mmmm = number to add to the stack pointer (mmmm divided by four equals to the number of system call arguments)

System Call Stub:

0000  b8nnnnnnnn      mov     eax,nnnnnnnn
0005  bassssssss      mov     edx,SharedUserData!SystemCallStub
000a  ff12            call    dword ptr \[edx\]
000c  c2mmmm          ret     mmmm
-or-
000c  c3              ret

ntdll!KiFastSystemCall:

0000  8bd4            mov     edx,esp
0002  0f34            sysenter
ntdll!KiFastSystemCallRet:
0004  c3              ret

Stack layout at the time of 'sysenter'

ESP+0x00  address of 'ret mmmm' or 'ret' in system call stub
   +0x04  caller of system call stub
   +0x08  1st syscall argument
   +0x0c  2nd syscall argument
   +0x10  3rd syscall argument
   +0x14  4th syscall argument
   +0x18  5th syscall argument
   +0x1c  6th syscall argument
   ...

WOW64

• caller calls system call stub
• system call stub calls wow64cpu!X86SwitchTo64BitMode
• wow64cpu!X86SwitchTo64BitMode jumps to wow64cpu!CpupReturnFromSimulatedCode
• wow64cpu!CpupReturnFromSimulatedCode sets up a new stack and jumps to a handler function, which will convert the 32 bit arguments to 64 bit and enter the kernel via SYSCALL
• nnnnnnnn = system call number
• fs:[0C0h] = contains address of wow64cpu!X86SwitchTo64BitMode
• mmmm = number to add to the stack pointer (mmmm divided by four equals to the number of system call arguments)
• ssssssss = address of wow64cpu!CpupReturnFromSimulatedCode
• uuuuuuuu = index into handler table, probably depends on the number and types of syscall arguments

System Call Stub 1:

0000  b8nnnnnnnn      mov     eax,nnnnnnnn
0005  33c9            xor     ecx,ecx
0007  8d542404        lea     edx,\[esp+4\]
000b  64ff15c0000000  call    dword ptr fs:\[0C0h\]
0012  83c404          add     esp,4
0015  c2mmmm          ret     mmmm
-or-
0015  c3              ret

System Call Stub 2:

0000  b8nnnnnnnn      mov     eax,nnnnnnnn
0005  b9uuuuuuuu      mov     ecx,uuuuuuuu
000a  8d542404        lea     edx,\[esp+4\]
000e  64ff15c0000000  call    dword ptr fs:\[0C0h\]
0015  83c404          add     esp,4
0018  c2mmmm          ret     mmmm
-or-
0018  c3              ret

wow64cpu!X86SwitchTo64BitMode:

0000  eassssssss3300  jmp     0033:ssssssss

Stack layout at the time of 'jmp 0033:ssssssss'

ESP+0x00  address of 'add esp,4' in system call stub
   +0x04  caller of system call stub
   +0x08  1st syscall argument
   +0x0c  2nd syscall argument
   +0x10  3rd syscall argument
   +0x14  4th syscall argument
   +0x18  5th syscall argument
   +0x1c  6th syscall argument
   ...

WIN64

• caller calls system call stub
• system call stub enters the kernel via SYSCALL
• number of system call arguments can not be derived from the system call stub
• nnnnnnnn = system call number

System Call Stub:

0000  4c8bd1          mov     r10,rcx
0003  b8nnnnnnnn      mov     eax,nnnnnnnn
0008  0f05            syscall
000a  c3              ret

Stack layout at the time of 'syscall'

RSP+0x00  caller of system call stub
   +0x08  reserved (1st syscall argument is in RCX)
   +0x10  reserved (2nd syscall argument is in RDX)
   +0x18  reserved (3rd syscall argument is in R8)
   +0x20  reserved (4th syscall argument is in R9)
   +0x28  5th syscall argument
   +0x30  6th syscall argument
   ...