Re: [Openvpn-devel] [PATCH] Feedback wanted: proof-of-concept recvmmsg() support

[David S. <op...@sf...>]

On 24/12/16 14:22, Steffan Karger wrote:
> This patch is not done, but I would like to get some early feedback
> because I'm not very familiar with this part of the code, nor with the
> APIs involved.  So I expect to have made some rooky mistakes.
> 
> A while back, I experimented a bit with recvmmsg(), to see if using it
> would yield some performance gain.  Simple tests (I don't have a good
> stress test setup handy) indicate that this improves throughput by a few
> percent.  I would expect more gain on the server side for P2MP servers
> though, where multiple client can be spamming the server at once.
> 
> What definitely still needs to be done:
>  * Improve error messages
>  * Improve documentation (doxygen, openvpn.8, Changes.rst)
>  * Figure out good default values
> 
> So, please, let me know what you think of the code.  And if you do have
> a good setup for performance testing, I'd love to hear what this change
> does for you.

Too bad I have changed employer ... as I have some vague memories doing
some recvmmsg() send even sendmmsg() testing on kernel-rt, when these
features were added there.  I'll meet some of my ex-colleagues later
this week and weekend (devconf.cz happens now) ... so I'll see if I can
get some help retrieving some reasonable test scenarios.

So to the patch ...

First of all, not all kernels carry these system calls, I believe they
were added in some of the 3.x kernels - but, IIRC, it has been
backported to at least the RHEL6 2.6.32 kernels.  My memory is scarce
about the 2.6.18 kernel base on RHEL5, but I believe only recvmmsg() is
supported there.

But kernel support isn't enough.  It needs to be supported in glibc as
well, where recvmmsg() arrived in glibc-2.12 and sendmmsg() arrived in
glibc-2.14.  And if glibc support is missing ... there's needed to so
some tweaks here, having a wrapper which does the proper syscall() with
SYS_recvmmsg and/or SYS_sendmmsg as the syscall number reference. (see
/usr/include/bits/syscall.h and man syscall(2) for more info) ... but
step carefully with such wrappers - as we need to consider
cross-platforms (not saying we should support this feature on all
platforms).

So all this considered, configure.ac will need to do some probing if
recvmmsg() and sendmmsg() are supported.

I know this patch covers recvmmsg() only.  But I think you can gain even
more performance if looking into sendmmsg() as well.

I'll try to play a bit with this patch and see how things go on my side
though.


-- 
kind regards,

David Sommerseth
OpenVPN Technologies, Inc