[uml-devel] Bad handling of invalif systemcalls

[Bodo S. <bst...@fu...>]

If a process in UML does a systemcall with the systemcall number
being less than 0, in TT-mode and SKAS-mode using SYSEMU, the
process is killed by an SIGTRAP instead of simply returning -ENOSYS.
In SKAS-mode without SYSEMU, UML even crashes, no matter if SYSEMU
is unsupported by the host or switched off in UML:

     Kernel panic - not syncing: handle_trap - failed to wait at end of syscall, errno = 4, status = 2943

The reason is, that UML can't distinguish between a debugger trap
and an systemcall interception. Currently, it checks the systemcall
number. If it is less than 0, it assumes the event to be a debugger
trap. It would be better to assume a debugger trap only, if the
syscall number is -1 (which it is guaranteed to be in case of a
debugger event), but even then UML wouldn't be safe. Syscalls with
syscall number -1 still would be a problem!

AFAICS, the only solution for this is using the PTRACE_O_TRACESYSGOOD
option. This option seems to be specific for linux, but the problem
maybe is specific for linux, too.

So, here attached are three patches. The first adds a check for
availability and function of
     ptrace(PTRACE_SETOPTIONS,,,PTRACE_O_TRACESYSGOOD) to the normal
ptrace checks.

The second implements the usage of the option in SKAS-mode.

The third does the same for TT-mode.

For the third patch I'm quite anxious, that there could go something
wrong when using the debugger. I don't understand much about this.
Maybe someone else could look into this?

Regards
Bodo