#76 Changes needed to port vde to solaris

[Brian Vandenberg]

The changes I've made to get this building are on an air-gapped
network and I don't have the means to provide them directly.

1) Add solaris/sunos as valid options to the [generated] configure script

2) Rename sockaddr_un instances named "sun" because a macro of that name
already exists for testing OS; the following may work, though you may
want to double check these changes to ensure comments or unrelated
code wasn't modified:

perl -pi -e 's{\bsun\b}{saddr_un}g' grep -lRIP --include='*.'{c,h} '\bsun\b'

3) Replace all uses of u_intXX_t with uintXX_t; the following is approximately
what I used to make the changes:

perl -pi -e 's{\bu_int(\d+)_t\b}{uint$1_t}g' grep -lRIP --include='*.'{c,h} '\bu_int\d+_t\b'

4) Change ifdef's of the following form:

if defined(VDE_DARWIN) || defined(VDE_FREEBSD)

... to:

if !defined( VDE_LINUX )

... in the following locations:

src/vde_l3/vde_buff.h:29
src/vde_l3/vde_l3.c:43
src/vde_pcapplug.c:47
src/vde_switch/tuntap.c:37
src/wirefilter.c:42

5) Add inclusions for <strings.h> anywhere index()/rindex() are used. In my change-set
I made these inclusions conditioned on the macro "sun" being defined, though I'm
guessing that wasn't strictly necessary.</strings.h>

Files changed:

src/vde_switch/tuntap.c
src/lib/libvdeplug.c
src/lib/libvdesnmp.c
src/vde_l3/vde_l3.c
src/vde_switch/hash.c
src/vde_switch/vde_switch.c

6) Add an inclusion in src/vde_over_ns/dns.c for <stdlib.h>; it's needed for alloca()</stdlib.h>

7) Rename s_addr in arp_header to something other than s_addr in the following files:

src/vde_l3/vde_buf.h
src/vde_l3/vde_l3.c

/usr/include/netinet/in.h defines a macro named s_addr that's used for accessing
elements of various system header-defined structs.

8) In src/vde_over_ns/vde_over_ns.c, LOG_PERROR isn't available in Solaris. I don't
see an appropriate alternative in the man page for openlog. I #ifdef'd it so
the solaris branch doesn't use LOG_PERROR, but I'm not sure what you'd want done here.

9) In src/wirefilter.c: the "timersub" macro isn't defined on solaris. I copied the
one from linux into wirefilter.c:

if defined( sun ) && !defined( timersub )

/.../

endif

10) I haven't [yet] implemented open_tap for solaris. I changed the following files:

src/vde_plug2tap.c
src/vde_switch/tuntap.c

... by adding the following:

if defined( VDE_SOLARIS )

int open_tap( char *dev ) {
assert( 0 && "open_tap not implemented for solaris" );
return 0;
}

endif

... and manually added a definition to config.h for VDE_SOLARIS to accomodate this change.
I'm not sure when I'll get open_tap implemented; hopefully soon.

11) daemon() doesn't exist in solaris. My implementation came
almost verbatim from examples I found on the internet, such as:

http://www.danielhall.me/2010/01/writing-a-daemon-in-c/

... where the only non-obvious implementation detail is how to portably redirect
stdin/stdout/stderr to /dev/null; I did it like this:

/ ... /
if( !noclose ) {
int fd = open( "/dev/null", O_RDWR );

// Redirects stdin/stdout/stderr to /dev/null
dup2( fd, 0 ); //stdin
dup2( fd, 1 ); //stdout
dup2( fd, 2 ); //stderr

if( fd > 2 ) {
  close( fd );
}

}

I added it to libvdecommon.a; the .c is wrapped in #if defined( sun ) to prevent it affecting
other operating systems, but I suppose it's possible to do something in the build to make
that conditionally built.

Furthermore, I added a header "vde_daemon.h" (#ifdef wrapped in the same way) and included
it in the following files:

src/dpipe.c
src/kvde_switch/consmgmt.c
src/slirpvde/slirpvde.c
src/vde_autolink.c
src/vde_over_ns/vde_over_ns.c
src/vde_pcapplug.c
src/vde_plug2tap.c
src/vde_switch/consmgmt.c
src/vdeq.c
src/wirefilter.c

12) open_memstream() is also not available on solaris; neither is funopen(). I read through the
source for libc for IllumOS (a fork of OpenSolaris before oracle shutdown OpenSolaris) and
found nothing resembling glibc's mechanisms for custom callbacks for overriding default
behavior for streams.

The only way to pull this off in a [relatively] clean way is with library interposition.  In
addition to interposition, I used macros of the form:

define PUBLIC attribute ((visibility ("default")))

define LOCAL attribute ((visibility ("hidden")))

... to prevent the interposed functions from being exported and interfering with code outside
of vde that links against vde.  A concern with this implementation: if a stream returned
from open_memstream() leaks to code outside of libvde, it'll use libc semantics for operating
on the stream.

open_memstream() uses tmpfile() to get a file stream, and does a bit of book-keeping.

In all, it was necessary to interpose functions in the following list, with some
extra logic added to call the libc implementation if the stream isn't from open_memstream():

  fprintf   - I wrote this to always forward the call to the interposed vfprintf
  vfprintf  - used asprintf() then the interposed fwrite() before freeing what asprintf() returned
  fputc     - just calls fwrite
  fwrite    - implements all the logic for writing to the buffer and expanding it with realloc()
  fclose    - nothing special

The functions use the following general pattern:

  // use of LOCAL prevents fwrite() from being exported from libvde
  LOCAL size_t fwrite( const void* ptr, size_t sz, size_t nitems, FILE* fs ) {
    static fwrite_fptr_t fptr;

    if( !fptr ) {
      // searches for a symbol of the same name as the current function
      // dynamically.  Since this function is marked hidden, it cannot be
      // found with dlsym() so it'll find the libc implementation.
      fptr = (typeof(fptr))dlsym( RTLD_DEFAULT, __FUNCTION__ );
    }

    if( !is_a_memstream( fs ) ) {
      return fptr( ptr, sz, nitems, fs );
    }
    /* ... */
  }

These are the only functions currently used in VDE to write into file streams allocated
by open_memstream.

The following fstream-related functions did not need interposition:

  fopen  - This doesn't take an fstream
  fread  - The current glibc implementation of memstreams never tags a stream as having
           data available for read, so fread() doesn't read and simply returns.

           With my implementation, the only way fread() could ever successfully read anything
           is if someone wrote to the stream using a function that wasn't interposed.


The only other piece that needs to be implemented is a way to track memstreams that have been
allocated.  I didn't go for a thread-safe implementation; it's a quick'n dirty proof of
concept that'll get improved as we make further use of it.  In short, I made a global array:

  struct fstream_struct_t {
    FILE     *current_file;         // the file stream being tracked
    size_t    current_length;       // amount of data that's been written into the buffer
    char     *write_buff;           // current write buffer
    size_t    write_buff_sz;        // current actual size of the write buffer
    char    **external_buff_ptr;    // points at the char* passed into open_memstream
    size_t   *external_buff_sz_ptr; // points at the size_t passed into open_memstream
  };

  static struct fstream_struct_t fstreams[32];

... and implemented a way to search it, etc.  Like I said, quick'n dirty.

[Brian Vandenberg]

Oh dear; I should've just attached that as a file or something.

I submitted a comment that isn't showing up. I'm editing to add it to this comment.

I configured with:

CPPFLAGS="-I/opt/csw/include" LDFLAGS="-Wl,-L/opt/csw/lib -Wl,-R/opt/csw/lib" LIBS="-lsocket -lnsl"

The LIBS thing is a necessity on solaris. You should probably add it to the build.

Stuff installed /opt/csw comes from the OpenCSW project. While not officially part of Solaris, I've yet to see a solaris install without /opt/csw (that wasn't stuck in the dark ages anyway). You'd probably be safe making the build check whether /opt/csw exists and adding those flags if it does exist.