[Audit-test-developer] [design] How to write / run / audit syscall wrappers?

[Jiri J. <jja...@re...>]

Hello Linda & others,

I've been doing some syscall work recently and while doing it,
I decided to "clean up" the do_ipc and do_socketcall wrappers, which
seemed like a duplicated functionality, since they call exactly the same
library functions as normal do_* wrappers. Also, I really wanted to get
rid of the ipc headerhack. :)
This was somewhat amplified by the fact that the code in ipc_common.c
was over-shared, meaning that ie. semget was using flags for semctl
or semop. So I made a series of ~6 commits, carefully moving all the
functionality from ipc_common.c into separate wrappers and removing the
do_ipc wrapper. I did the same for do_socketcall, which just calls bind,
using a library function, like do_bind. All this with removing
respective sections from syscalls/*.conf, of course.

I was quite happy with the series, since - functionality wise - it was
transparent. However when I tested it on MODE=32, the syscalls bucket
started throwing ERRORs. Some investigation uncovered that the syscalls
bucket was actually using these "duplicated" wrappers for proper
auditing - because auditctl works with real syscalls, not libc
functions. The extra wrappers were therefore nothing more than a name,
simplifying logic in the syscalls bucket.

This goes against some other approaches used in the suite - in the
network bucket, for example, which - based on the architecture - selects
proper syscall name for auditctl, while still calling the original
syscall wrapper (which uses library functions).

-----------------------------------------------------------------------

This led me into a certain design question I'd like to ask here; how to
design syscall wrappers and the execution and auditing infrastructure
around them? What would be the best approach?

I've identified 3 most obvious ways to write a syscall wrapper:

  A) use syscall(__NR_syscallname, ...) directly, bypassing libc
  B) use libc functions
  C) use (A), but simulate libc using #ifdefs manually

These approaches solve the "which syscalls to run where" problem
somewhat differently and therefore have different benefits and
drawbacks in various situations:

  1) compile time

    (A) is a bit problematic, since we would need to come up with a full
    logic of what syscalls to compile on what architectures. We already
    do that on some level, but this option calls for a separate mapping
    file (or so) instead of simple Makefile-based conditions,
    integrating it somehow into the make system.

    (B) and (C) are really easy - the libc (or custom #ifdefs) take care
    of which syscall should be called on which architecture. (C) has
    the disadvantage of actually doing the mapping from (A), just in
    a less visible way.

  2) run (execution) time

    (A) again needs to re-use the mapping file (or logic) in most cases,
    since - if we want to call ie. open-like syscall, we need to call
    explicitly either do_open (non-arm) or do_openat (arm)

    (B) and (C) simplify this case a lot, but may fall short if we
    *really* want to call openat instead of open on non-arm

  3) auditing time

   (A) shines here, IMO, since the auditctl mapping is 1:1 to the do_*
   wrappers. It still needs to re-use the mapping file (and therefore
   needs to manually specify which syscalls to audit on which archs),
   but it's very straightforward and clear.

   (B) and (C) have a significant problem here - we don't know which
   syscalls are being called "under the hood" of the do_* wrappers, so
   we need to try them out and then create per-arch hacks in the code
   similar to what we've seen in the arm patchset recently, ie.
   "set up auditing for fchmodat and call do_chmod", which can be
   somewhat confusing. The other option is to duplicate wrappers like
   do_socketcall, but the per-arch hacks still persist. We could create
   a mapping file for them, but then we might as well use (A).


So what would be the best approach for new (and existing, over time)
syscall wrappers?

I personally really like (A) due to its clean design - there are no
"Note: There is no glibc wrapper for this system call" exceptions
and it's clear what syscalls run on which architectures and 32/64bit
variations. The mapping file, with its helper bash functions doing ie.
"is this syscall relevant for current arch/mode?" or "list all relevant
syscalls for current arch/mode", along with some documentation, should
be mostly easy to implement.
A practical example in the syscalls bucket would be checking for arch
relevancy in the `+' function with no per-arch or per-mode conditions
in the .conf files. The rollup log (or run.bash --list) would then show
which syscalls were actually run.

This also means that I would have to throw away most of my series on
do_ipc and do_socketcall, possibly re-implementing them in the future,
but I'm fine with that.

What's your opinion?

Thanks,
Jiri


PS: I'm asking because we currently have ~70 new syscall wrappers staged
    for review and there's not much consistency in terms of (A) vs (B)
    vs (C).